[PATCH 00/27] tests/tcg: run tests with meson

[PATCH 00/27] tests/tcg: run tests with meson

[Pierrick Bouvier]

This series is an experiment to replace TCG tests makefiles with meson, with the
goal of having something more readable and maintainable in the long term.
Also it closes the gap that tcg tests had with the rest of tests, by integrating
them with meson directly.

It covers all existing tests for aarch64 architecture, including
user/system/multiarch-user/multiarch-system. I'll carry the effort to port all
other architectures once we agree on a base. We agreed with Alex to not merge
anything before all arch are covered.

Series first declare the global infrastructure, then add aarch64 and multiarch
tests, and finally plugin tests.

For review, I would suggest to start with aarch64 patches (end of the series) to
get an idea of what they will look like, before the implementation itself. You
can also compare that to existing Makefiles, they follow the same order.

It implements all requirements listed on original thread [1], except cross
container support, which can be easily added on top once we agree on the base.
[1] https://lore.kernel.org/qemu-devel/87wlwfsay4.fsf@draig.linaro.org/

We also have for free:
- correct and complete dependencies for any test, including:
  scripts, c.inc, headers, reference files and binaries/plugins.
- catch test declaration issues at configure time vs test time.
- proper data types for variables instead of string expansion with make.
- we could now build tests binaries by default with 'all' target.
- hopefully, better readability to your taste.

Main ideas for the current design are:
- make arch files as simple and explicit as possible. If it conflicts,
  always pick explicit, as long as we don't expose meson details.
  remove any kind of "skip, override, filter" logic.
- never expose custom_target and meson details in arch files.
- catch any spelling mistake on test or any dependency.
- keep complexity contained in tests/tcg/meson.build.

Usage:

```
make check-tcg
make check-tcg-aarch64-linux-user
make check-tcg-aarch64-softmmu

from build folder:
./pyvenv/bin/meson test --list --suite tcg
ninja clean
./pyvenv/bin/meson test aarch64-softmmu-asid2 --verbose
```

Pierrick Bouvier (27):
  tests/tcg/multiarch/system/memory.c: remove unused variable
  tests/tcg/multiarch/plugin/check-plugin-output.sh: take test output as
    input
  tests/tcg/multiarch/plugin: rename check-plugin-output to
    regex-compare
  tests/tcg: introduce meson.build
  tests/tcg/meson.build: introduce exe_name
  tests/tcg/meson.build: introduce test_name
  tests/tcg/meson.build: introduce cflags
  tests/tcg/meson.build: introduce qemu_args
  tests/tcg/meson.build: introduce env_var
  tests/tcg/plugins: build list of test_plugins
  tests/tcg/meson.build: introduce plugin_test
  tests/tcg/meson.build: test gdb support and introduce gdb_arch
  tests/tcg/meson.build: introduce gdb_test
  tests/tcg/meson.build: add default flag for testing compiler support
  tests/tcg/meson.build: introduce wrapper
  tests/tcg/meson.build: introduce expected_output
  tests/tcg/meson.build: add wrapper run_and_check_forbidden_output
  tests/tcg/meson.build: add wrapper run_with_input
  tests/tcg/meson.build: add wrapper record_replay
  tests/tcg/meson.build: add wrapper check_plugin_output
  tests/tcg/aarch64: add user tests
  tests/tcg/aarch64: add system tests
  tests/tcg/multiarch: declare user tests
  tests/tcg/aarch64: add multiarch user tests
  tests/tcg/multiarch: declare system tests
  tests/tcg/aarch64: add multiarch system tests
  tests/tcg/meson.build: add generic plugin tests

 tests/meson.build                             |   2 +-
 tests/tcg/aarch64/meson.build                 | 248 ++++++++++++++++
 tests/tcg/aarch64/system/meson.build          | 100 +++++++
 tests/tcg/meson.build                         | 276 ++++++++++++++++++
 tests/tcg/multiarch/Makefile.target           |   3 +-
 tests/tcg/multiarch/meson.build               | 149 ++++++++++
 .../multiarch/plugin/check-plugin-output.sh   |  36 ---
 tests/tcg/multiarch/plugin/regex-compare.sh   |  28 ++
 tests/tcg/multiarch/sha1.ref                  |   1 +
 tests/tcg/multiarch/system/memory.c           |   3 +-
 tests/tcg/multiarch/system/meson.build        |  53 ++++
 tests/tcg/plugins/meson.build                 |  11 +-
 tests/tcg/scripts/check_plugin_output.sh      |  19 ++
 tests/tcg/scripts/record_replay.sh            |  16 +
 .../scripts/run_and_check_forbidden_output.sh |  19 ++
 tests/tcg/scripts/run_and_diff.sh             |  12 +
 tests/tcg/scripts/run_with_input.sh           |  12 +
 17 files changed, 943 insertions(+), 45 deletions(-)
 create mode 100644 tests/tcg/aarch64/meson.build
 create mode 100644 tests/tcg/aarch64/system/meson.build
 create mode 100644 tests/tcg/meson.build
 create mode 100644 tests/tcg/multiarch/meson.build
 delete mode 100755 tests/tcg/multiarch/plugin/check-plugin-output.sh
 create mode 100755 tests/tcg/multiarch/plugin/regex-compare.sh
 create mode 100644 tests/tcg/multiarch/sha1.ref
 create mode 100644 tests/tcg/multiarch/system/meson.build
 create mode 100755 tests/tcg/scripts/check_plugin_output.sh
 create mode 100755 tests/tcg/scripts/record_replay.sh
 create mode 100755 tests/tcg/scripts/run_and_check_forbidden_output.sh
 create mode 100755 tests/tcg/scripts/run_and_diff.sh
 create mode 100755 tests/tcg/scripts/run_with_input.sh

-- 
2.43.0

[PATCH 01/27] tests/tcg/multiarch/system/memory.c: remove unused variable

[Pierrick Bouvier]

Caught this since by compiling this file with -Wall, -Werror.

../tests/tcg/aarch64/system/../../multiarch/system/memory.c: In function ‘init_test_data_s8’:
../tests/tcg/aarch64/system/../../multiarch/system/memory.c:97:18: error: unused variable ‘bottom’ [-Werror=unused-variable]
   97 |     uint8_t top, bottom, *ptr = &test_data[0];
      |                  ^~~~~~
../tests/tcg/aarch64/system/../../multiarch/system/memory.c:97:13: error: unused variable ‘top’ [-Werror=unused-variable]
   97 |     uint8_t top, bottom, *ptr = &test_data[0];
      |             ^~~
../tests/tcg/aarch64/system/../../multiarch/system/memory.c: In function ‘read_test_data_s8’:
../tests/tcg/aarch64/system/../../multiarch/system/memory.c:384:14: error: unused variable ‘ok’ [-Werror=unused-variable]
  384 |         bool ok;
      |

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/multiarch/system/memory.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/tests/tcg/multiarch/system/memory.c b/tests/tcg/multiarch/system/memory.c
index 7508f6b916d..adc7ac60c1f 100644
--- a/tests/tcg/multiarch/system/memory.c
+++ b/tests/tcg/multiarch/system/memory.c
@@ -94,7 +94,7 @@ static inline uint8_t get_byte(int index, bool neg)
 
 static void init_test_data_s8(bool neg_first)
 {
-    uint8_t top, bottom, *ptr = &test_data[0];
+    uint8_t *ptr = &test_data[0];
     int i;
 
     ml_printf("Filling test area with s8 pairs (%s):",
@@ -381,7 +381,6 @@ static bool read_test_data_s8(int offset, bool neg_first)
 
     for (i = 0; i < max; i++) {
         int16_t first, second;
-        bool ok;
         first = *ptr++;
         second = *ptr++;
 
-- 
2.43.0

[PATCH 02/27] tests/tcg/multiarch/plugin/check-plugin-output.sh: take test output as input

[Pierrick Bouvier]

Instead of executing test, just read it's output from a file. This
allows to standardize how to call check plugins scripts.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/multiarch/Makefile.target           |  3 +--
 .../multiarch/plugin/check-plugin-output.sh   | 22 ++++++-------------
 2 files changed, 8 insertions(+), 17 deletions(-)

diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index 508149d57bd..f46983cc76e 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -211,8 +211,7 @@ run-plugin-test-plugin-mem-access-with-libmem.so: \
 	PLUGIN_ARGS=$(COMMA)print-accesses=true
 run-plugin-test-plugin-mem-access-with-libmem.so: \
 	CHECK_PLUGIN_OUTPUT_COMMAND= \
-	$(SRC_PATH)/tests/tcg/multiarch/plugin/check-plugin-output.sh \
-	$(QEMU) $<
+	$(SRC_PATH)/tests/tcg/multiarch/plugin/check-plugin-output.sh $@.out
 run-plugin-test-plugin-syscall-filter-with-libsyscall.so:
 run-plugin-test-plugin-set-pc-with-libsetpc.so:
 
diff --git a/tests/tcg/multiarch/plugin/check-plugin-output.sh b/tests/tcg/multiarch/plugin/check-plugin-output.sh
index 80607f04b5d..338c9ced23e 100755
--- a/tests/tcg/multiarch/plugin/check-plugin-output.sh
+++ b/tests/tcg/multiarch/plugin/check-plugin-output.sh
@@ -1,8 +1,7 @@
 #!/usr/bin/env bash
 
-# This script runs a given executable using qemu, and compare its standard
-# output with an expected plugin output.
-# Each line of output is searched (as a regexp) in the expected plugin output.
+# This script checks that plugin_output contains line from test_output.
+# Each line of output is searched (as a regexp) in plugin_output.
 
 set -euo pipefail
 
@@ -19,18 +18,11 @@ check()
     grep "$pattern" "$file" > /dev/null || die "\"$pattern\" not found in $file"
 }
 
-[ $# -eq 3 ] || die "usage: qemu_bin exe plugin_out_file"
+[ $# -eq 2 ] || die "usage: test_output plugin_out"
 
-qemu_bin=$1; shift
-exe=$1;shift
-plugin_out=$1; shift
+test_output=$1;shift
+plugin_output=$1;shift
 
-expected()
-{
-    $qemu_bin $exe ||
-        die "running $exe failed"
-}
-
-expected | while read line; do
-    check "$plugin_out" "$line"
+cat $test_output | while read line; do
+    check "$plugin_output" "$line"
 done
-- 
2.43.0

[PATCH 03/27] tests/tcg/multiarch/plugin: rename check-plugin-output to regex-compare

[Pierrick Bouvier]

We'll reuse this name later in the series, so change existing script.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/multiarch/Makefile.target                             | 2 +-
 .../plugin/{check-plugin-output.sh => regex-compare.sh}         | 0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename tests/tcg/multiarch/plugin/{check-plugin-output.sh => regex-compare.sh} (100%)

diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index f46983cc76e..6907c2830aa 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -211,7 +211,7 @@ run-plugin-test-plugin-mem-access-with-libmem.so: \
 	PLUGIN_ARGS=$(COMMA)print-accesses=true
 run-plugin-test-plugin-mem-access-with-libmem.so: \
 	CHECK_PLUGIN_OUTPUT_COMMAND= \
-	$(SRC_PATH)/tests/tcg/multiarch/plugin/check-plugin-output.sh $@.out
+	$(SRC_PATH)/tests/tcg/multiarch/plugin/regex-compare.sh $@.out
 run-plugin-test-plugin-syscall-filter-with-libsyscall.so:
 run-plugin-test-plugin-set-pc-with-libsetpc.so:
 
diff --git a/tests/tcg/multiarch/plugin/check-plugin-output.sh b/tests/tcg/multiarch/plugin/regex-compare.sh
similarity index 100%
rename from tests/tcg/multiarch/plugin/check-plugin-output.sh
rename to tests/tcg/multiarch/plugin/regex-compare.sh
-- 
2.43.0

[PATCH 04/27] tests/tcg: introduce meson.build

[Pierrick Bouvier]

Every target declare its tests, and add them in global tcg_tests
dictionary, indexed by target name. See comment for full description.

We make sure that if a user repeats a test several time it won't result
in duplicated executables or tests. In next commits, we'll introduce
what's needed to deal with that.

A test is compiled from a single source file (and later, a single set of
compile flags). We make sure dependencies are correctly handled by
generating a depfile.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/meson.build     |  2 +-
 tests/tcg/meson.build | 94 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 95 insertions(+), 1 deletion(-)
 create mode 100644 tests/tcg/meson.build

diff --git a/tests/meson.build b/tests/meson.build
index 9ba04bbedd3..5877d2f60b1 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -74,7 +74,7 @@ subdir('decode')
 
 if 'CONFIG_TCG' in config_all_accel
   subdir('fp')
-  subdir('tcg/plugins')
+  subdir('tcg')
 endif
 
 subdir('audio')
diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
new file mode 100644
index 00000000000..72d1a9f8986
--- /dev/null
+++ b/tests/tcg/meson.build
@@ -0,0 +1,94 @@
+env = find_program('env')
+
+tcg_tests = {}
+# tcg_tests is a dictionary with following structure:
+#
+# {
+#   'name_of_target': {
+#     'cc': cross_compiler,
+#     'folder': 'folder_for_test_src_files',
+#     'qemu': emulators['qemu-...'],
+#     'tests': [
+#       {
+#         'src_file': {
+#         }
+#       },
+#       ...
+#     ]
+#   }
+# }
+#
+# Every test executable, is built only once.
+# Tests for a given src use the same executable by default, and their definition
+# is guaranteed to be unique also.
+# Default name is derived from src.
+
+# plugins come first, as we need to build the list
+subdir('plugins')
+
+# Finally, we can create all test executables and test targets
+foreach target, plan: tcg_tests
+  # Detect duplicated executables/tests, and report an error to force user to
+  # choose how to deal with it.
+  built_tests = {}
+  added_tests = {}
+
+  # return a clear error if user mispell a target entry
+  foreach key, _ : plan
+    if key not in ['cc', 'folder', 'qemu', 'tests']
+      error('unknown tcg test plan entry \'' + key + '\' for target ' + target)
+    endif
+  endforeach
+
+  cc = plan['cc']
+  folder = plan['folder']
+  qemu = plan['qemu']
+  tests = plan['tests']
+
+  foreach t : tests
+    foreach src, setup: t
+      # meson '/' operator drops left operand if right is an absolute path
+      src = folder / src
+      file = files(src)
+      test = fs.name(file)
+      exe_name = fs.stem(src)
+
+      exe_name = target + '-' + exe_name
+      test_name = exe_name
+
+      if test_name in added_tests
+        error('test ' + test_name + ' was already added: ' +
+              'specify a different \'test_name\'')
+      endif
+
+      # build executable if needed
+      if exe_name not in built_tests
+        exe = custom_target(exe_name,
+                            input: file,
+                            command: [cc, '@INPUT@',
+                                      '-o', '@OUTPUT@', '-static',
+                                      '-MMD', '-MF', '@DEPFILE@',
+                                      '-Wall', '-Werror',
+                                      '-O0', '-g', '-fno-strict-aliasing',
+                                      ],
+                            depfile: exe_name + '.d',
+                            output: exe_name + '.test',
+                            build_by_default: false)
+        built_tests += {exe_name: exe}
+      endif
+
+      depends = []
+
+      exe = built_tests[exe_name]
+      cmd = [qemu, exe]
+
+      # some tests expect QEMU env var to be set
+      cmd = ['QEMU=' + qemu.full_path(), cmd]
+
+      test(test_name, env, args: cmd,
+           depends: depends,
+           suite: ['tcg', 'tcg-' + target])
+      added_tests += {test_name: true}
+    endforeach
+  endforeach
+endforeach
-- 
2.43.0

[PATCH 05/27] tests/tcg/meson.build: introduce exe_name

[Pierrick Bouvier]

We add possibility to duplicate a test executable for a given source
file. This is needed when different cflags need to be used (like
vectorized vs non-vectorized for instance, like sve tests in aarch64).

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 72d1a9f8986..2c6d85c586a 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -11,6 +11,7 @@ tcg_tests = {}
 #     'tests': [
 #       {
 #         'src_file': {
+#           'exe_name': ['provide an alternative binary name'],
 #         }
 #       },
 #       ...
@@ -18,10 +19,10 @@ tcg_tests = {}
 #   }
 # }
 #
-# Every test executable, is built only once.
+# Every test executable, identified by 'exe_name' is built only once.
 # Tests for a given src use the same executable by default, and their definition
 # is guaranteed to be unique also.
-# Default name is derived from src.
+# Default name is derived from src if 'exe_name' is omitted.
 
 # plugins come first, as we need to build the list
 subdir('plugins')
@@ -47,11 +48,23 @@ foreach target, plan: tcg_tests
 
   foreach t : tests
     foreach src, setup: t
+      # return a clear error if user mispell a setup entry
+      foreach key, _ : setup
+        if key not in [
+                       'exe_name',
+                       ]
+          error('unknown tcg setup entry \'' + key + '\' for test ' + src)
+        endif
+      endforeach
+
       # meson '/' operator drops left operand if right is an absolute path
       src = folder / src
       file = files(src)
       test = fs.name(file)
       exe_name = fs.stem(src)
+      if 'exe_name' in setup
+        exe_name = setup['exe_name']
+      endif
 
       exe_name = target + '-' + exe_name
       test_name = exe_name
-- 
2.43.0

[PATCH 06/27] tests/tcg/meson.build: introduce test_name

[Pierrick Bouvier]

We add possibility to duplicate a test for a given src. This allows to
declare several tests that still share the same executable, thus saving
space and compile time.

A side note to mention that meson perfectly supports declaring several
tests with the same name. However, we prevent that to force dev to
clarify what is the intent of each individual test.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 2c6d85c586a..bb0184732fd 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -12,6 +12,7 @@ tcg_tests = {}
 #       {
 #         'src_file': {
 #           'exe_name': ['provide an alternative binary name'],
+#           'test_name': ['provide an alternative test name'],
 #         }
 #       },
 #       ...
@@ -21,8 +22,8 @@ tcg_tests = {}
 #
 # Every test executable, identified by 'exe_name' is built only once.
 # Tests for a given src use the same executable by default, and their definition
-# is guaranteed to be unique also.
-# Default name is derived from src if 'exe_name' is omitted.
+# is guaranteed to be unique also. They can be duplicated by setting 'test_name'.
+# Default name is derived from src if 'exe_name' and 'test_name' are omitted.
 
 # plugins come first, as we need to build the list
 subdir('plugins')
@@ -51,7 +52,7 @@ foreach target, plan: tcg_tests
       # return a clear error if user mispell a setup entry
       foreach key, _ : setup
         if key not in [
-                       'exe_name',
+                       'exe_name', 'test_name',
                        ]
           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
         endif
@@ -66,8 +67,13 @@ foreach target, plan: tcg_tests
         exe_name = setup['exe_name']
       endif
 
-      exe_name = target + '-' + exe_name
       test_name = exe_name
+      if 'test_name' in setup
+        test_name = setup['test_name']
+      endif
+
+      exe_name = target + '-' + exe_name
+      test_name = target + '-' + test_name
 
       if test_name in added_tests
         error('test ' + test_name + ' was already added: ' +
-- 
2.43.0

[PATCH 07/27] tests/tcg/meson.build: introduce cflags

[Pierrick Bouvier]

We add possibility to declare cflags for a given test. Since we want to
preserve the uniqueness of test binaries, we prevent repeating those
cflags later, for an existing binary.

This ensures one executable == one set of cflags, and if dev needs
duplication, they can use a custom 'exe_name' to override this.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index bb0184732fd..ad56fdf4ab5 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -13,6 +13,7 @@ tcg_tests = {}
 #         'src_file': {
 #           'exe_name': ['provide an alternative binary name'],
 #           'test_name': ['provide an alternative test name'],
+#           'cflags': ['cflags to compile test', ...],
 #         }
 #       },
 #       ...
@@ -20,7 +21,8 @@ tcg_tests = {}
 #   }
 # }
 #
-# Every test executable, identified by 'exe_name' is built only once.
+# Every test executable, identified by 'exe_name' is built only once, and we
+# ensure cflags are specified only once, on first declaration.
 # Tests for a given src use the same executable by default, and their definition
 # is guaranteed to be unique also. They can be duplicated by setting 'test_name'.
 # Default name is derived from src if 'exe_name' and 'test_name' are omitted.
@@ -51,7 +53,7 @@ foreach target, plan: tcg_tests
     foreach src, setup: t
       # return a clear error if user mispell a setup entry
       foreach key, _ : setup
-        if key not in [
+        if key not in ['cflags',
                        'exe_name', 'test_name',
                        ]
           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
@@ -80,6 +82,15 @@ foreach target, plan: tcg_tests
               'specify a different \'test_name\'')
       endif
 
+      cflags = []
+      if 'cflags' in setup
+        if exe_name in built_tests
+          error('test ' + exe_name + ' was already built with cflags: ' +
+                'remove cflags or set \'exe_name\' (if cflags are different)')
+        endif
+        cflags = setup['cflags']
+      endif
+
       # build executable if needed
       if exe_name not in built_tests
         exe = custom_target(exe_name,
@@ -89,7 +100,7 @@ foreach target, plan: tcg_tests
                                       '-MMD', '-MF', '@DEPFILE@',
                                       '-Wall', '-Werror',
                                       '-O0', '-g', '-fno-strict-aliasing',
-                                      ],
+                                      cflags],
                             depfile: exe_name + '.d',
                             output: exe_name + '.test',
                             build_by_default: false)
-- 
2.43.0

[PATCH 08/27] tests/tcg/meson.build: introduce qemu_args

[Pierrick Bouvier]

We add possibility to use custom arguments when calling qemu for a test.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index ad56fdf4ab5..28a7e529c7e 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -14,6 +14,7 @@ tcg_tests = {}
 #           'exe_name': ['provide an alternative binary name'],
 #           'test_name': ['provide an alternative test name'],
 #           'cflags': ['cflags to compile test', ...],
+#           'qemu_args': ['qemu command line flags', ...],
 #         }
 #       },
 #       ...
@@ -53,7 +54,7 @@ foreach target, plan: tcg_tests
     foreach src, setup: t
       # return a clear error if user mispell a setup entry
       foreach key, _ : setup
-        if key not in ['cflags',
+        if key not in ['cflags', 'qemu_args',
                        'exe_name', 'test_name',
                        ]
           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
@@ -108,9 +109,13 @@ foreach target, plan: tcg_tests
       endif
 
       depends = []
+      qemu_args = []
+      if 'qemu_args' in setup
+        qemu_args = setup['qemu_args']
+      endif
 
       exe = built_tests[exe_name]
-      cmd = [qemu, exe]
+      cmd = [qemu, qemu_args, exe]
 
       # some tests expect QEMU env var to be set
       cmd = ['QEMU=' + qemu.full_path(), cmd]
-- 
2.43.0

[PATCH 09/27] tests/tcg/meson.build: introduce env_var

[Pierrick Bouvier]

We add possibility to set an environment variable when running a test.
An example of a test requiring this is
tests/tcg/multiarch/linux/linux-sigrtminmax.c.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 28a7e529c7e..b6a375e9eee 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -15,6 +15,7 @@ tcg_tests = {}
 #           'test_name': ['provide an alternative test name'],
 #           'cflags': ['cflags to compile test', ...],
 #           'qemu_args': ['qemu command line flags', ...],
+#           'env_var': ['VAR=value', ...],
 #         }
 #       },
 #       ...
@@ -55,7 +56,7 @@ foreach target, plan: tcg_tests
       # return a clear error if user mispell a setup entry
       foreach key, _ : setup
         if key not in ['cflags', 'qemu_args',
-                       'exe_name', 'test_name',
+                       'exe_name', 'test_name', 'env_var',
                        ]
           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
         endif
@@ -120,6 +121,10 @@ foreach target, plan: tcg_tests
       # some tests expect QEMU env var to be set
       cmd = ['QEMU=' + qemu.full_path(), cmd]
 
+      if 'env_var' in setup
+        cmd = [setup['env_var'], cmd]
+      endif
+
       test(test_name, env, args: cmd,
            depends: depends,
            suite: ['tcg', 'tcg-' + target])
-- 
2.43.0

[PATCH 10/27] tests/tcg/plugins: build list of test_plugins

[Pierrick Bouvier]

We previously had list of test_plugins sources, but not the target
themselves. Create this list, so tests can then reference a plugin by
its name from there.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build         |  1 +
 tests/tcg/plugins/meson.build | 11 +++++++----
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index b6a375e9eee..5cd80f3c030 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -30,6 +30,7 @@ tcg_tests = {}
 # Default name is derived from src if 'exe_name' and 'test_name' are omitted.
 
 # plugins come first, as we need to build the list
+test_plugins = {}
 subdir('plugins')
 
 # Finally, we can create all test executables and test targets
diff --git a/tests/tcg/plugins/meson.build b/tests/tcg/plugins/meson.build
index d7f8f0ae0ad..2abc317d1e7 100644
--- a/tests/tcg/plugins/meson.build
+++ b/tests/tcg/plugins/meson.build
@@ -1,4 +1,4 @@
-test_plugins = [
+test_plugins_src = [
 'bb.c',
 'discons.c',
 'empty.c',
@@ -14,9 +14,12 @@ test_plugins = [
 
 t = []
 if get_option('plugins')
-  foreach i : test_plugins
-    t += shared_module(fs.stem(i), files(i),
-                       dependencies: plugins_deps)
+  foreach i : test_plugins_src
+    name = fs.stem(i)
+    plugin = shared_module(name, files(i),
+                           dependencies: plugins_deps)
+    t += plugin
+    test_plugins += {name: plugin}
   endforeach
 endif
 if t.length() > 0
-- 
2.43.0

[PATCH 11/27] tests/tcg/meson.build: introduce plugin_test

[Pierrick Bouvier]

We add possibility to declare a test that is based on a plugin.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 40 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 39 insertions(+), 1 deletion(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 5cd80f3c030..a47f1ada779 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -16,6 +16,10 @@ tcg_tests = {}
 #           'cflags': ['cflags to compile test', ...],
 #           'qemu_args': ['qemu command line flags', ...],
 #           'env_var': ['VAR=value', ...],
+#           'plugin_test': {
+#             'plugin': 'plugin_name',
+#             'args': ['plugin_args', ...],
+#           },
 #         }
 #       },
 #       ...
@@ -57,12 +61,25 @@ foreach target, plan: tcg_tests
       # return a clear error if user mispell a setup entry
       foreach key, _ : setup
         if key not in ['cflags', 'qemu_args',
-                       'exe_name', 'test_name', 'env_var',
+                       'exe_name', 'test_name', 'env_var', 'plugin_test',
                        ]
           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
         endif
       endforeach
 
+      # return a clear error if user mispell a plugin entry
+      if 'plugin_test' in setup
+        foreach key, _ : setup['plugin_test']
+          if key not in ['plugin', 'args']
+            error('unknown tcg plugin entry \'' + key + '\' for test ' + src)
+          endif
+        endforeach
+      endif
+
+      if 'plugin_test' in setup and not get_option('plugins')
+        continue
+      endif
+
       # meson '/' operator drops left operand if right is an absolute path
       src = folder / src
       file = files(src)
@@ -76,6 +93,12 @@ foreach target, plan: tcg_tests
       if 'test_name' in setup
         test_name = setup['test_name']
       endif
+      if 'plugin_test' in setup
+        # remove lib prefix
+        plugin = test_plugins[setup['plugin_test']['plugin']]
+        plugin_name = fs.stem(plugin).substring(3)
+        test_name = 'plugin-' + plugin_name + '-with-' + test_name
+      endif
 
       exe_name = target + '-' + exe_name
       test_name = target + '-' + test_name
@@ -116,6 +139,21 @@ foreach target, plan: tcg_tests
         qemu_args = setup['qemu_args']
       endif
 
+      if 'plugin_test' in setup
+        plugin_setup = setup['plugin_test']
+        plugin = test_plugins[plugin_setup['plugin']]
+        plugin_args = []
+        if 'args' in plugin_setup
+          # add empty arg to generate an additional comma
+          plugin_args = ['', plugin_setup['args']]
+        endif
+        plugin_args = ','.join(plugin_args)
+
+        # since we turn plugin into a string, add dependency explicitly
+        depends += plugin
+        qemu_args = ['-plugin', plugin.full_path() + plugin_args, qemu_args]
+      endif
+
       exe = built_tests[exe_name]
       cmd = [qemu, qemu_args, exe]
 
-- 
2.43.0

[PATCH 12/27] tests/tcg/meson.build: test gdb support and introduce gdb_arch

[Pierrick Bouvier]

A given target can now declare which gdb architecture it requires.
Also, we add the code to check gdb capabilities, and version.
Tests can use gdb_version.version_compare('>=X.Y') to test this.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index a47f1ada779..1496cba460c 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -1,5 +1,11 @@
 env = find_program('env')
 
+gdb_progs = ['gdb-multiarch', 'gdb']
+if config_host.has_key('GDB')
+  gdb_progs = [config_host['GDB'], gdb_progs]
+endif
+gdb = find_program(gdb_progs, required: false)
+
 tcg_tests = {}
 # tcg_tests is a dictionary with following structure:
 #
@@ -7,6 +13,7 @@ tcg_tests = {}
 #   'name_of_target': {
 #     'cc': cross_compiler,
 #     'folder': 'folder_for_test_src_files',
+#     'gdb_arch': 'gdb architecture requirement',
 #     'qemu': emulators['qemu-...'],
 #     'tests': [
 #       {
@@ -33,6 +40,18 @@ tcg_tests = {}
 # is guaranteed to be unique also. They can be duplicated by setting 'test_name'.
 # Default name is derived from src if 'exe_name' and 'test_name' are omitted.
 
+gdb_arch_supported = {}
+gdb_version = '0'
+if gdb.found()
+  gdb_version = gdb.version()
+  gdb_arch = run_command(python, files('../../scripts/probe-gdb-support.py'),
+                           gdb,
+                           check: true)
+  foreach arch: gdb_arch.stdout().split()
+    gdb_arch_supported += {arch: true}
+  endforeach
+endif
+
 # plugins come first, as we need to build the list
 test_plugins = {}
 subdir('plugins')
@@ -46,13 +65,14 @@ foreach target, plan: tcg_tests
 
   # return a clear error if user mispell a target entry
   foreach key, _ : plan
-    if key not in ['cc', 'folder', 'qemu', 'tests']
+    if key not in ['cc', 'folder', 'gdb_arch', 'qemu', 'tests']
       error('unknown tcg test plan entry \'' + key + '\' for target ' + target)
     endif
   endforeach
 
   cc = plan['cc']
   folder = plan['folder']
+  gdb_arch = plan['gdb_arch']
   qemu = plan['qemu']
   tests = plan['tests']
 
-- 
2.43.0

[PATCH 13/27] tests/tcg/meson.build: introduce gdb_test

[Pierrick Bouvier]

We can now declare a test requiring gdb interaction, based on
tests/guest-debug/run-test.py.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 1496cba460c..d0080eb0f9f 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -5,6 +5,7 @@ if config_host.has_key('GDB')
   gdb_progs = [config_host['GDB'], gdb_progs]
 endif
 gdb = find_program(gdb_progs, required: false)
+prog_gdb_test = find_program('../guest-debug/run-test.py')
 
 tcg_tests = {}
 # tcg_tests is a dictionary with following structure:
@@ -22,6 +23,7 @@ tcg_tests = {}
 #           'test_name': ['provide an alternative test name'],
 #           'cflags': ['cflags to compile test', ...],
 #           'qemu_args': ['qemu command line flags', ...],
+#           'gdb_test': ['gdb test args'],
 #           'env_var': ['VAR=value', ...],
 #           'plugin_test': {
 #             'plugin': 'plugin_name',
@@ -80,7 +82,7 @@ foreach target, plan: tcg_tests
     foreach src, setup: t
       # return a clear error if user mispell a setup entry
       foreach key, _ : setup
-        if key not in ['cflags', 'qemu_args',
+        if key not in ['cflags', 'qemu_args', 'gdb_test',
                        'exe_name', 'test_name', 'env_var', 'plugin_test',
                        ]
           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
@@ -96,6 +98,10 @@ foreach target, plan: tcg_tests
         endforeach
       endif
 
+      if 'gdb_test' in setup and gdb_arch not in gdb_arch_supported
+        continue
+      endif
+
       if 'plugin_test' in setup and not get_option('plugins')
         continue
       endif
@@ -113,7 +119,9 @@ foreach target, plan: tcg_tests
       if 'test_name' in setup
         test_name = setup['test_name']
       endif
-      if 'plugin_test' in setup
+      if 'gdb_test' in setup
+        test_name = 'gdb-' + test_name
+      elif 'plugin_test' in setup
         # remove lib prefix
         plugin = test_plugins[setup['plugin_test']['plugin']]
         plugin_name = fs.stem(plugin).substring(3)
@@ -177,6 +185,16 @@ foreach target, plan: tcg_tests
       exe = built_tests[exe_name]
       cmd = [qemu, qemu_args, exe]
 
+      if 'gdb_test' in setup
+        cmd = [prog_gdb_test,
+               '--gdb', gdb,
+               '--qemu', qemu,
+               '--qargs', ' '.join(qemu_args),
+               '--bin', exe,
+               setup['gdb_test'],
+              ]
+      endif
+
       # some tests expect QEMU env var to be set
       cmd = ['QEMU=' + qemu.full_path(), cmd]
 
-- 
2.43.0

[PATCH 14/27] tests/tcg/meson.build: add default flag for testing compiler support

[Pierrick Bouvier]

This global variable can be used as cflags when checking if a compiler
supports specific flags.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index d0080eb0f9f..aca59933d71 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -1,3 +1,4 @@
+cc_check_args = ['-xc', '/dev/null', '-o', '/dev/null', '-c']
 env = find_program('env')
 
 gdb_progs = ['gdb-multiarch', 'gdb']
-- 
2.43.0

[PATCH 15/27] tests/tcg/meson.build: introduce wrapper

[Pierrick Bouvier]

We add possibility to wrap a test command through a given script. This
can be used to provide any kind of ad-hoc check for specific tests.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index aca59933d71..5380a360830 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -26,6 +26,7 @@ tcg_tests = {}
 #           'qemu_args': ['qemu command line flags', ...],
 #           'gdb_test': ['gdb test args'],
 #           'env_var': ['VAR=value', ...],
+#           'wrapper': [program_to_launch_test, 'args'...],
 #           'plugin_test': {
 #             'plugin': 'plugin_name',
 #             'args': ['plugin_args', ...],
@@ -85,7 +86,7 @@ foreach target, plan: tcg_tests
       foreach key, _ : setup
         if key not in ['cflags', 'qemu_args', 'gdb_test',
                        'exe_name', 'test_name', 'env_var', 'plugin_test',
-                       ]
+                       'wrapper']
           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
         endif
       endforeach
@@ -196,6 +197,10 @@ foreach target, plan: tcg_tests
               ]
       endif
 
+      if 'wrapper' in setup
+        cmd = [setup['wrapper'], cmd]
+      endif
+
       # some tests expect QEMU env var to be set
       cmd = ['QEMU=' + qemu.full_path(), cmd]
 
-- 
2.43.0

[PATCH 16/27] tests/tcg/meson.build: introduce expected_output

[Pierrick Bouvier]

A test can declare it expects a specific output on stdout. In this case,
we compare it to a reference file. We implement this instead of relying
on a wrapper because we need the flexibility to resolve name of
reference file per target.

For instance, multiarch float tests expect different ref files for each
target, which is something we can't express with a test wrapper.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build             |  9 ++++++++-
 tests/tcg/scripts/run_and_diff.sh | 12 ++++++++++++
 2 files changed, 20 insertions(+), 1 deletion(-)
 create mode 100755 tests/tcg/scripts/run_and_diff.sh

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 5380a360830..3d2c0d6583b 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -7,6 +7,7 @@ if config_host.has_key('GDB')
 endif
 gdb = find_program(gdb_progs, required: false)
 prog_gdb_test = find_program('../guest-debug/run-test.py')
+prog_run_and_diff = find_program('./scripts/run_and_diff.sh')
 
 tcg_tests = {}
 # tcg_tests is a dictionary with following structure:
@@ -24,6 +25,7 @@ tcg_tests = {}
 #           'test_name': ['provide an alternative test name'],
 #           'cflags': ['cflags to compile test', ...],
 #           'qemu_args': ['qemu command line flags', ...],
+#           'expected_output': 'file_to_compare_output',
 #           'gdb_test': ['gdb test args'],
 #           'env_var': ['VAR=value', ...],
 #           'wrapper': [program_to_launch_test, 'args'...],
@@ -84,7 +86,7 @@ foreach target, plan: tcg_tests
     foreach src, setup: t
       # return a clear error if user mispell a setup entry
       foreach key, _ : setup
-        if key not in ['cflags', 'qemu_args', 'gdb_test',
+        if key not in ['cflags', 'qemu_args', 'expected_output', 'gdb_test',
                        'exe_name', 'test_name', 'env_var', 'plugin_test',
                        'wrapper']
           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
@@ -197,6 +199,11 @@ foreach target, plan: tcg_tests
               ]
       endif
 
+      if 'expected_output' in setup
+        expected = files(folder / setup['expected_output'])
+        cmd = [prog_run_and_diff, expected, cmd]
+      endif
+
       if 'wrapper' in setup
         cmd = [setup['wrapper'], cmd]
       endif
diff --git a/tests/tcg/scripts/run_and_diff.sh b/tests/tcg/scripts/run_and_diff.sh
new file mode 100755
index 00000000000..d1e9243c49d
--- /dev/null
+++ b/tests/tcg/scripts/run_and_diff.sh
@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+set -euo pipefail
+
+if [ $# -lt 2 ]; then
+    echo "run_and_diff: expected_output_file cmd [args]..." 1>&2
+    exit 1
+fi
+expected="$1";shift
+set -x
+diff <("$@") $expected
-- 
2.43.0

[PATCH 17/27] tests/tcg/meson.build: add wrapper run_and_check_forbidden_output

[Pierrick Bouvier]

One of our test requires to make sure output does not contain a specific
string. An example is system multiarch test run-gdbstub-untimely-packet.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build                         |  3 +++
 .../scripts/run_and_check_forbidden_output.sh | 19 +++++++++++++++++++
 2 files changed, 22 insertions(+)
 create mode 100755 tests/tcg/scripts/run_and_check_forbidden_output.sh

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 3d2c0d6583b..f379449e582 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -8,6 +8,9 @@ endif
 gdb = find_program(gdb_progs, required: false)
 prog_gdb_test = find_program('../guest-debug/run-test.py')
 prog_run_and_diff = find_program('./scripts/run_and_diff.sh')
+prog_run_and_check_forbidden_output = find_program(
+  './scripts/run_and_check_forbidden_output.sh'
+)
 
 tcg_tests = {}
 # tcg_tests is a dictionary with following structure:
diff --git a/tests/tcg/scripts/run_and_check_forbidden_output.sh b/tests/tcg/scripts/run_and_check_forbidden_output.sh
new file mode 100755
index 00000000000..4486ca8ef83
--- /dev/null
+++ b/tests/tcg/scripts/run_and_check_forbidden_output.sh
@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+set -euo pipefail
+
+if [ $# -lt 2 ]; then
+    echo "run_and_check_forbidden_output: 'forbidden string' cmd [args]..." 1>&2
+    exit 1
+fi
+forbidden="$1";shift
+output=$("$@" 2>&1)
+set -x
+if ! echo "$output" | grep -Fi "$forbidden"
+then
+    echo "output does not contain forbidden string: $forbidden"
+    exit 0
+fi
+echo "found forbidden string: $forbidden" 1>&2
+exit 1
-- 
2.43.0

[PATCH 18/27] tests/tcg/meson.build: add wrapper run_with_input

[Pierrick Bouvier]

Semihosting console tests require console interaction, and were flagged
as manual because of this. This wrapper automates interaction, by
sending a specific string on stdin.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build               |  1 +
 tests/tcg/scripts/run_with_input.sh | 12 ++++++++++++
 2 files changed, 13 insertions(+)
 create mode 100755 tests/tcg/scripts/run_with_input.sh

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index f379449e582..7201a7809e1 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -8,6 +8,7 @@ endif
 gdb = find_program(gdb_progs, required: false)
 prog_gdb_test = find_program('../guest-debug/run-test.py')
 prog_run_and_diff = find_program('./scripts/run_and_diff.sh')
+prog_run_with_input = find_program('./scripts/run_with_input.sh')
 prog_run_and_check_forbidden_output = find_program(
   './scripts/run_and_check_forbidden_output.sh'
 )
diff --git a/tests/tcg/scripts/run_with_input.sh b/tests/tcg/scripts/run_with_input.sh
new file mode 100755
index 00000000000..3d280547c3a
--- /dev/null
+++ b/tests/tcg/scripts/run_with_input.sh
@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+set -euo pipefail
+
+if [ $# -lt 2 ]; then
+    echo "run_with_input: input_string cmd [args]..." 1>&2
+    exit 1
+fi
+input="$1";shift
+set -x
+echo "$input" | "$@"
-- 
2.43.0

[PATCH 19/27] tests/tcg/meson.build: add wrapper record_replay

[Pierrick Bouvier]

Runs a record and replay test, and make sure the output produced by
both runs is the same.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build              |  1 +
 tests/tcg/scripts/record_replay.sh | 16 ++++++++++++++++
 2 files changed, 17 insertions(+)
 create mode 100755 tests/tcg/scripts/record_replay.sh

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 7201a7809e1..f0efd46ee3e 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -7,6 +7,7 @@ if config_host.has_key('GDB')
 endif
 gdb = find_program(gdb_progs, required: false)
 prog_gdb_test = find_program('../guest-debug/run-test.py')
+prog_record_replay = find_program('./scripts/record_replay.sh')
 prog_run_and_diff = find_program('./scripts/run_and_diff.sh')
 prog_run_with_input = find_program('./scripts/run_with_input.sh')
 prog_run_and_check_forbidden_output = find_program(
diff --git a/tests/tcg/scripts/record_replay.sh b/tests/tcg/scripts/record_replay.sh
new file mode 100755
index 00000000000..57775a2d4c1
--- /dev/null
+++ b/tests/tcg/scripts/record_replay.sh
@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+set -euo pipefail
+
+if [ $# -lt 2 ]; then
+    echo "record_replay: qemu_bin [args]..." 1>&2
+    exit 1
+fi
+qemu="$1";shift
+tmp=$(mktemp -d)
+trap "rm -rf $tmp" EXIT
+set -x
+$qemu -icount shift=5,rr=record,rrfile=$tmp/rr.bin "$@" |& tee $tmp/record
+$qemu -icount shift=5,rr=replay,rrfile=$tmp/rr.bin "$@" |& tee $tmp/replay
+diff $tmp/record $tmp/replay
-- 
2.43.0

[PATCH 20/27] tests/tcg/meson.build: add wrapper check_plugin_output

[Pierrick Bouvier]

This wrapper runs a qemu command, record command output and plugin
output, and forward both to a check script.
We use this pattern in several plugin tests (for instance, with
validate-memory-counts.py).

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build                    |  1 +
 tests/tcg/scripts/check_plugin_output.sh | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+)
 create mode 100755 tests/tcg/scripts/check_plugin_output.sh

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index f0efd46ee3e..6e46c121fda 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -6,6 +6,7 @@ if config_host.has_key('GDB')
   gdb_progs = [config_host['GDB'], gdb_progs]
 endif
 gdb = find_program(gdb_progs, required: false)
+prog_check_plugin_output = find_program('./scripts/check_plugin_output.sh')
 prog_gdb_test = find_program('../guest-debug/run-test.py')
 prog_record_replay = find_program('./scripts/record_replay.sh')
 prog_run_and_diff = find_program('./scripts/run_and_diff.sh')
diff --git a/tests/tcg/scripts/check_plugin_output.sh b/tests/tcg/scripts/check_plugin_output.sh
new file mode 100755
index 00000000000..4d12e23ad60
--- /dev/null
+++ b/tests/tcg/scripts/check_plugin_output.sh
@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+set -euo pipefail
+
+if [ $# -lt 3 ]; then
+    echo "check_plugin_output: check_cmd qemu_bin [args]..." 1>&2
+    exit 1
+fi
+check_cmd="$1"; shift
+qemu="$1"; shift
+tmp=$(mktemp -d)
+trap "rm -rf $tmp" EXIT
+
+set -x
+
+$qemu -d plugin -D $tmp/plugin "$@" |& tee $tmp/output
+cat $tmp/plugin
+$check_cmd $tmp/output $tmp/plugin
-- 
2.43.0

[PATCH 21/27] tests/tcg/aarch64: add user tests

[Pierrick Bouvier]

We are finally ready to add tests. Let's start with aarch64 user first.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/aarch64/meson.build | 213 ++++++++++++++++++++++++++++++++++
 tests/tcg/meson.build         |   3 +
 2 files changed, 216 insertions(+)
 create mode 100644 tests/tcg/aarch64/meson.build

diff --git a/tests/tcg/aarch64/meson.build b/tests/tcg/aarch64/meson.build
new file mode 100644
index 00000000000..9c42b129551
--- /dev/null
+++ b/tests/tcg/aarch64/meson.build
@@ -0,0 +1,213 @@
+cc = find_program('aarch64-linux-gnu-gcc', required : false)
+
+if not cc.found()
+  subdir_done()
+endif
+
+feat_cflags = {
+  'aes': '-march=armv8-a+aes',
+  'sve': '-march=armv8.1-a+sve',
+  'sve2': '-march=armv8.1-a+sve2',
+  'v8.2': '-march=armv8.2-a',
+  'v8.3': '-march=armv8.3-a',
+  'v8.5': '-march=armv8.5-a',
+  'bti':  '-mbranch-protection=standard',
+  'mte':  '-march=armv8.5-a+memtag',
+  'sme':  '-Wa,-march=armv9-a+sme',
+}
+cc_has_feat = {}
+
+foreach feat, flags : feat_cflags
+  cmd = run_command([cc, cc_check_args, flags], check: false)
+  avail = cmd.returncode() == 0
+  cc_has_feat += {feat: avail}
+  if not avail
+    warning('aarch64 tcg tests for feature \'' + feat + '\' ' +
+            'disabled: unsupported by cross-compiler')
+  endif
+endforeach
+
+gdb_feat_version = {
+  'sme-tiles': '14.1',
+  'mte': '15.1',
+  'mte-baremetal': '16.0',
+}
+gdb_has_feat = {}
+
+foreach feat, min_ver : gdb_feat_version
+  avail = gdb_version.version_compare('>=' + min_ver)
+  gdb_has_feat += {feat: avail}
+  if not avail
+    warning('aarch64 gdb tests for feature \'' + feat + '\' ' +
+            'disabled: unsupported by gdb')
+  endif
+endforeach
+
+if 'qemu-aarch64' not in emulators
+  subdir_done()
+endif
+
+tests = []
+
+# Base architecture tests
+tests += {
+  '../arm/fcvt.c': {'cflags': '-lm', 'expected_output': 'fcvt.ref'},
+  'pcalign-a64.c': {},
+  'lse2-fault.c': {},
+  'test-2248.c': {},
+  'test-2150.c': {},
+}
+
+if cc_has_feat['v8.2']
+  tests += {'dcpop.c': {'cflags': feat_cflags['v8.2']}}
+endif
+
+if cc_has_feat['v8.5']
+  tests += {'dcpodp.c': {'cflags': feat_cflags['v8.5']}}
+endif
+
+# Pauth Tests
+if cc_has_feat['v8.3']
+  f = feat_cflags['v8.3']
+  cpu_max = ['-cpu', 'max']
+  cpu_neoverse = ['-cpu', 'neoverse-v1']
+  tests += {
+    'test-2375.c': {'cflags': f},
+    'pauth-1.c': {'cflags': f, 'qemu_args': cpu_max},
+    'pauth-2.c': {'cflags': f, 'qemu_args': cpu_max},
+    # Choose a cpu with FEAT_Pauth but without FEAT_FPAC for pauth-[45].
+    'pauth-4.c': {'cflags': f, 'qemu_args': cpu_neoverse},
+    'pauth-5.c': {'cflags': f, 'qemu_args': cpu_neoverse},
+  }
+endif
+
+# BTI Tests
+# bti-1 tests the elf notes, so we require special compiler support.
+if cc_has_feat['bti']
+  f = [feat_cflags['bti'], '-fno-stack-protector', '-nostdlib']
+  tests += {
+    'bti-1.c': {'cflags': f},
+    'bti-3.c': {'cflags': f},
+  }
+endif
+# bti-2 tests PROT_BTI, so no special compiler support required.
+tests += {'bti-2.c': {}}
+
+# MTE Tests
+if cc_has_feat['mte']
+  f = feat_cflags['mte']
+  tests += {
+    'mte-1.c': {'cflags': f},
+    'mte-2.c': {'cflags': f},
+    'mte-3.c': {'cflags': f},
+    'mte-4.c': {'cflags': f},
+    'mte-5.c': {'cflags': f},
+    'mte-6.c': {'cflags': f},
+    'mte-7.c': {'cflags': f},
+    'mte-8.c': {'cflags': f},
+  }
+endif
+
+# SME Tests
+if cc_has_feat['sme']
+  f = feat_cflags['sme']
+  tests += {
+    'sme-outprod1.c': {'cflags': f},
+    'sme-smopa-1.c': {'cflags': f},
+    'sme-smopa-2.c': {'cflags': f},
+    'sme-fmopa-1.c': {'cflags': f},
+    'sme-fmopa-2.c': {'cflags': f},
+    'sme-fmopa-3.c': {'cflags': f},
+  }
+endif
+
+# GCS Tests
+tests += {
+  'gcsstr.c': {},
+  'gcspushm.c': {},
+  'gcsss.c': {},
+}
+
+# System Registers Tests
+tests += {'sysregs.c': {}}
+
+# Crypto Tests
+tests += {'test-aes.c': {'cflags': feat_cflags['aes']}}
+
+# SVE Tests
+if cc_has_feat['sve']
+  f = feat_cflags['sve']
+  tests += {
+    'sve-ioctls.c': {'cflags': f},
+    'sve-str.c': {'cflags': [f, '-O1']},
+  }
+endif
+
+# SVE2 Tests
+if cc_has_feat['sve2']
+  tests += {'test-826.c': {'cflags': feat_cflags['sve2']}}
+endif
+
+# GDB Tests
+if cc_has_feat['sve']
+  f = feat_cflags['sve']
+  tests += {
+    'sysregs.c': {
+      'exe_name': 'sysregs-sve',
+      'cflags': f,
+      'gdb_test': ['--test', files('gdbstub/test-sve.py')],
+    },
+    'sve-ioctls.c': {
+      'gdb_test': ['--test', files('gdbstub/test-sve-ioctl.py')],
+    },
+  }
+endif
+
+if cc_has_feat['sme']
+  f = feat_cflags['sme']
+  sysregs_sme = 'sysregs-sme'
+  tests += {
+    'sysregs.c': {
+      'exe_name': sysregs_sme,
+      'cflags': f,
+      'gdb_test': ['--test', files('gdbstub/test-sme.py'), '--',
+                   'test_sme', '--gdb_basic_za_test'],
+    }
+  }
+  if gdb_has_feat['sme-tiles']
+    tests += {
+      'sysregs.c': {
+        'test_name': 'sysregs-sme-tile-slice',
+        'exe_name': sysregs_sme,
+        'gdb_test': ['--test', files('gdbstub/test-sme.py'), '--',
+                     'test_sme', '--gdb_tile_slice_test'],
+      }
+    }
+  endif
+  tests += {
+    'sysregs.c': {
+      'test_name': 'sysregs-sme2',
+      'exe_name': sysregs_sme,
+      'gdb_test': ['--test', files('gdbstub/test-sme2.py')],
+    }
+  }
+endif
+
+if cc_has_feat['mte'] and gdb_has_feat['mte']
+  f = feat_cflags['mte']
+  tests += {
+    'mte-8.c': {
+      'gdb_test': ['--test', files('gdbstub/test-mte.py'), '--', '--mode=user'],
+    }
+  }
+endif
+
+tcg_tests += {
+  'aarch64-linux-user': {
+    'cc': cc,
+    'folder': 'aarch64',
+    'gdb_arch': 'aarch64',
+    'qemu': emulators['qemu-aarch64'],
+    'tests': tests,
+  }
+}
diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 6e46c121fda..8633c58459b 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -68,6 +68,9 @@ endif
 test_plugins = {}
 subdir('plugins')
 
+# Now let's go through all architectures
+subdir('aarch64')
+
 # Finally, we can create all test executables and test targets
 foreach target, plan: tcg_tests
   # Detect duplicated executables/tests, and report an error to force user to
-- 
2.43.0

[PATCH 22/27] tests/tcg/aarch64: add system tests

[Pierrick Bouvier]

Let's continue with system tests. Take note of how we don't generate
intermediate (common) objects, but instead provide a single build
command for every test, embedding all dependencies.

Besides reducing boilerplate, this makes sure we track correctly all the
dependencies for a given test. Compilation time difference is invisible.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/aarch64/meson.build        |  2 +
 tests/tcg/aarch64/system/meson.build | 69 ++++++++++++++++++++++++++++
 2 files changed, 71 insertions(+)
 create mode 100644 tests/tcg/aarch64/system/meson.build

diff --git a/tests/tcg/aarch64/meson.build b/tests/tcg/aarch64/meson.build
index 9c42b129551..e6cf5309649 100644
--- a/tests/tcg/aarch64/meson.build
+++ b/tests/tcg/aarch64/meson.build
@@ -43,6 +43,8 @@ foreach feat, min_ver : gdb_feat_version
   endif
 endforeach
 
+subdir('system')
+
 if 'qemu-aarch64' not in emulators
   subdir_done()
 endif
diff --git a/tests/tcg/aarch64/system/meson.build b/tests/tcg/aarch64/system/meson.build
new file mode 100644
index 00000000000..a1edf1b8442
--- /dev/null
+++ b/tests/tcg/aarch64/system/meson.build
@@ -0,0 +1,69 @@
+if 'qemu-system-aarch64' not in emulators
+  subdir_done()
+endif
+
+tests = []
+
+minilib_dir = meson.current_source_dir() / '..' / '..' / 'minilib'
+minilib_printf = files(minilib_dir / 'printf.c')[0]
+link_script = files('kernel.ld')[0]
+boot = files('boot.S')[0]
+cflags = ['-nostdlib',
+          '-Wa,--noexecstack',
+          '-I', minilib_dir, minilib_printf, boot,
+          '-Wl,-T', link_script]
+qemu_base_args = ['-display', 'none',
+                  '-semihosting-config', 'enable=on',
+                  '-kernel']
+qemu_def_args = ['-M', 'virt', '-cpu', 'max', qemu_base_args]
+
+tests += {
+  'asid2.c': {'cflags': cflags, 'qemu_args': qemu_def_args},
+  'feat-xs.c': {'cflags': cflags, 'qemu_args': qemu_def_args},
+  'rme_gdi.c': {'cflags': cflags, 'qemu_args': qemu_def_args},
+  'semiheap.c': {'cflags': cflags, 'qemu_args': qemu_def_args},
+  'semiconsole.c': {
+    'cflags': cflags,
+    'qemu_args': ['-serial', 'none', '-chardev', 'stdio,mux=on,id=stdio0',
+                  '-semihosting-config', 'enable=on,chardev=stdio0',
+                  '-mon', 'chardev=stdio0,mode=readline',
+                  qemu_def_args],
+    'wrapper': [prog_run_with_input, 'X']
+  },
+  'vtimer.c': {
+    'cflags': cflags,
+    'qemu_args': ['-M', 'virt,virtualization=on,gic-version=2',
+                  '-cpu', 'cortex-a57', '-smp', '4',
+                  '-semihosting-config', 'enable=on,arg=2',
+                  qemu_base_args]
+  },
+}
+
+if cc_has_feat['v8.3']
+  tests += {
+    'pauth-3.c': {
+      'cflags': [feat_cflags['v8.3'], cflags],
+      'qemu_args': ['-M', 'virt', '-cpu', 'max,pauth-qarma5=on', qemu_base_args],
+    }
+  }
+endif
+
+if cc_has_feat['mte'] and gdb_has_feat['mte-baremetal']
+  tests += {
+    'mte.S': {
+      'cflags': [feat_cflags['mte'], cflags],
+      'qemu_args': ['-M', 'virt,mte=on', '-cpu', 'max', qemu_base_args],
+      'gdb_test': ['--test', files('../gdbstub/test-mte.py'), '--', '--mode=system'],
+    }
+  }
+endif
+
+tcg_tests += {
+  'aarch64-softmmu': {
+    'cc': cc,
+    'folder': 'aarch64/system',
+    'gdb_arch': 'aarch64',
+    'qemu': emulators['qemu-system-aarch64'],
+    'tests': tests,
+  }
+}
-- 
2.43.0

[PATCH 23/27] tests/tcg/multiarch: declare user tests

[Pierrick Bouvier]

At this point, those tests are just declared and not used yet.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build           |   7 ++
 tests/tcg/multiarch/meson.build | 147 ++++++++++++++++++++++++++++++++
 tests/tcg/multiarch/sha1.ref    |   1 +
 3 files changed, 155 insertions(+)
 create mode 100644 tests/tcg/multiarch/meson.build
 create mode 100644 tests/tcg/multiarch/sha1.ref

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 8633c58459b..9ec33739884 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -67,12 +67,19 @@ endif
 # plugins come first, as we need to build the list
 test_plugins = {}
 subdir('plugins')
+# multiarch comes second, so it can be referenced by other arch after
+subdir('multiarch')
 
 # Now let's go through all architectures
 subdir('aarch64')
 
 # Finally, we can create all test executables and test targets
 foreach target, plan: tcg_tests
+  if target.startswith('multiarch-')
+    # those tests are included by architecture files if needed
+    continue
+  endif
+
   # Detect duplicated executables/tests, and report an error to force user to
   # choose how to deal with it.
   built_tests = {}
diff --git a/tests/tcg/multiarch/meson.build b/tests/tcg/multiarch/meson.build
new file mode 100644
index 00000000000..99af41b3fcc
--- /dev/null
+++ b/tests/tcg/multiarch/meson.build
@@ -0,0 +1,147 @@
+tests = []
+
+float_helpers = files('libs/float_helpers.c')[0]
+float_cflags = [float_helpers, '-lm']
+
+# GCC versions 12/13/14/15 at least incorrectly complain about
+# "'SHA1Transform' reading 64 bytes from a region of size 0"; see the gcc bug
+# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106709
+# Since this is just a standard piece of library code we've borrowed for a
+# TCG test case, suppress the warning rather than trying to modify the
+# code to work around the compiler.
+sha1_cflags = ['-Wno-stringop-overread', '-Wno-unknown-warning-option']
+
+# multiarch src path are written to be included from any other arch folder,
+# so all arch can directly add those tests without any modification.
+multiarch = meson.current_source_dir()
+
+tests += {
+  multiarch/'catch-syscalls.c': {},
+  multiarch/'float_convd.c': {
+    'cflags': float_cflags,
+    'expected_output': 'float_convd.ref',
+  },
+  multiarch/'float_convs.c': {
+    'cflags': float_cflags,
+    'expected_output': 'float_convs.ref',
+  },
+  multiarch/'float_madds.c': {
+    'cflags': float_cflags,
+    'expected_output': 'float_madds.ref',
+  },
+  multiarch/'fnmsub.c': {'cflags': ['-lm']},
+  multiarch/'follow-fork-mode.c': {},
+  multiarch/'late-attach.c': {},
+  multiarch/'munmap-pthread.c': {'cflags': ['-lpthread']},
+  multiarch/'overflow.c': {},
+  multiarch/'prot-none.c': {},
+  multiarch/'segfault.c': {},
+  multiarch/'sha1.c': {
+    'cflags': sha1_cflags,
+    'expected_output': multiarch/'sha1.ref'
+  },
+  multiarch/'sha512.c': {},
+  multiarch/'sigbus.c': {},
+  multiarch/'signals.c': {'cflags': ['-lrt', '-lpthread']},
+  multiarch/'sigreturn-sigmask.c': {'cflags': ['-lpthread']},
+  multiarch/'tb-link.c': {'cflags': ['-lpthread']},
+  multiarch/'test-mmap.c': {},
+  multiarch/'testthread.c': {'cflags': ['-lpthread']},
+  multiarch/'threadcount.c': {'cflags': ['-lpthread']},
+}
+
+# Linux tests
+tests += {
+  multiarch/'linux/linux-madvise.c': {},
+  multiarch/'linux/linux-shmat-maps.c': {},
+  multiarch/'linux/linux-shmat-null.c': {},
+  multiarch/'linux/linux-sigrtminmax.c': {},
+  multiarch/'linux/linux-test.c': {},
+  multiarch/'linux/test-vma.c': {},
+}
+
+# GDB tests
+tests += {
+  multiarch/'sha1.c': {
+    'gdb_test': ['--test', files('gdbstub/sha1.py')],
+  },
+}
+tests += {
+  multiarch/'sha1.c': {
+    'test_name': 'qxfer-auxv-read',
+    'gdb_test': ['--test', files('gdbstub/test-qxfer-auxv-read.py')],
+  },
+}
+tests += {
+  multiarch/'sha1.c': {
+    'test_name': 'proc-mappings',
+    'gdb_test': ['--test', files('gdbstub/test-proc-mappings.py')],
+  },
+}
+tests += {
+  multiarch/'segfault.c': {
+    'test_name': 'qxfer-siginfo-read',
+    'gdb_test': ['--test', files('gdbstub/test-qxfer-siginfo-read.py')],
+  },
+  multiarch/'testthread.c': {
+    'test_name': 'thread-breakpoint',
+    'gdb_test': ['--test', files('gdbstub/test-thread-breakpoint.py')],
+  },
+  multiarch/'sha512.c': {
+    'test_name': 'registers',
+    'gdb_test': ['--test', files('gdbstub/registers.py')],
+  },
+  multiarch/'prot-none.c': {
+    'gdb_test': ['--test', files('gdbstub/prot-none.py')],
+    'env_var': 'PROT_NONE_PY=1',
+  },
+  multiarch/'catch-syscalls.c': {
+    'gdb_test': ['--test', files('gdbstub/catch-syscalls.py')],
+  },
+  multiarch/'late-attach.c': {
+    'gdb_test': ['--no-suspend', '--test', files('gdbstub/late-attach.py')],
+    'env_var': 'LATE_ATTACH_PY=1',
+  },
+}
+
+tests += {
+  multiarch/'follow-fork-mode.c': {
+    'test_name': 'follow-fork-mode-child',
+    'gdb_test': ['--test', files('gdbstub/follow-fork-mode-child.py')],
+  },
+}
+tests += {
+  multiarch/'follow-fork-mode.c': {
+    'test_name': 'follow-fork-mode-parent',
+    'gdb_test': ['--test', files('gdbstub/follow-fork-mode-parent.py')],
+  },
+}
+
+# Specific plugin tests
+# Test plugin memory access instrumentation
+tests += {
+  multiarch/'plugin/test-plugin-mem-access.c': {
+    'plugin_test': {
+      'plugin': 'mem',
+      'args': ['print-accesses=true']
+    },
+    'wrapper': [prog_check_plugin_output,
+                find_program('plugin/regex-compare.sh')],
+  },
+  multiarch/'plugin/test-plugin-set-pc.c': {
+    'plugin_test': {
+      'plugin': 'setpc',
+    }
+  },
+  multiarch/'test-plugin-syscall-filter.c': {
+    'plugin_test': {
+      'plugin': 'syscall',
+    }
+  },
+}
+
+tcg_tests += {
+  'multiarch-linux-user': {
+    'tests': tests
+  }
+}
diff --git a/tests/tcg/multiarch/sha1.ref b/tests/tcg/multiarch/sha1.ref
new file mode 100644
index 00000000000..4a8a114634f
--- /dev/null
+++ b/tests/tcg/multiarch/sha1.ref
@@ -0,0 +1 @@
+SHA1=15dd99a1991e0b3826fede3deffc1feba42278e6
-- 
2.43.0

[PATCH 24/27] tests/tcg/aarch64: add multiarch user tests

[Pierrick Bouvier]

We can now add multiarch tests. We also take care of semihosting tests
here, by including them explicitly, instead of relying on any kind of
"per-target" config.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/aarch64/meson.build | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/tests/tcg/aarch64/meson.build b/tests/tcg/aarch64/meson.build
index e6cf5309649..53085278155 100644
--- a/tests/tcg/aarch64/meson.build
+++ b/tests/tcg/aarch64/meson.build
@@ -51,6 +51,9 @@ endif
 
 tests = []
 
+# Multi arch tests
+tests += tcg_tests['multiarch-linux-user']['tests']
+
 # Base architecture tests
 tests += {
   '../arm/fcvt.c': {'cflags': '-lm', 'expected_output': 'fcvt.ref'},
@@ -136,11 +139,29 @@ tests += {'sysregs.c': {}}
 # Crypto Tests
 tests += {'test-aes.c': {'cflags': feat_cflags['aes']}}
 
+# Vector SHA1
+# Work around compiler false-positive warning, as we do for the 'sha1' test
+tests += {'../multiarch/sha1.c': {
+            'exe_name': 'sha1-vector',
+            'cflags': ['-O3', '-Wno-stringop-overread'],
+            'expected_output': '../multiarch/sha1.ref',
+         }}
+
+# Vector versions of sha512 (-O3 triggers vectorisation)
+tests += {'../multiarch/sha512.c': {
+            'exe_name': 'sha512-vector',
+            'cflags': ['-O3'],
+         }}
+
 # SVE Tests
 if cc_has_feat['sve']
   f = feat_cflags['sve']
   tests += {
     'sve-ioctls.c': {'cflags': f},
+    '../multiarch/sha512.c': {
+       'exe_name': 'sha512-sve',
+       'cflags': f,
+    },
     'sve-str.c': {'cflags': [f, '-O1']},
   }
 endif
@@ -204,6 +225,18 @@ if cc_has_feat['mte'] and gdb_has_feat['mte']
   }
 endif
 
+# Semihosting tests
+# Add -I path for semicall.h
+semihosting_cflags = ['-I', meson.current_source_dir()]
+
+tests += {
+  '../multiarch/arm-compat-semi/semiconsole.c': {
+    'cflags': semihosting_cflags,
+    'wrapper': [prog_run_with_input, 'X'],
+  },
+  '../multiarch/arm-compat-semi/semihosting.c': {'cflags': semihosting_cflags},
+}
+
 tcg_tests += {
   'aarch64-linux-user': {
     'cc': cc,
-- 
2.43.0

[PATCH 25/27] tests/tcg/multiarch: declare system tests

[Pierrick Bouvier]

We now declare all multiarch system tests.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/multiarch/meson.build        |  2 +
 tests/tcg/multiarch/system/meson.build | 53 ++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)
 create mode 100644 tests/tcg/multiarch/system/meson.build

diff --git a/tests/tcg/multiarch/meson.build b/tests/tcg/multiarch/meson.build
index 99af41b3fcc..508fdb585b7 100644
--- a/tests/tcg/multiarch/meson.build
+++ b/tests/tcg/multiarch/meson.build
@@ -1,3 +1,5 @@
+subdir('system')
+
 tests = []
 
 float_helpers = files('libs/float_helpers.c')[0]
diff --git a/tests/tcg/multiarch/system/meson.build b/tests/tcg/multiarch/system/meson.build
new file mode 100644
index 00000000000..f71c12db38e
--- /dev/null
+++ b/tests/tcg/multiarch/system/meson.build
@@ -0,0 +1,53 @@
+tests = []
+
+# multiarch src path are written to be included from any other arch folder,
+# so all arch can directly add those tests.
+multiarch = meson.current_source_dir()
+
+tests += {
+  multiarch/'hello.c': {},
+  multiarch/'interrupt.c': {},
+  multiarch/'memory.c': {},
+}
+
+tests += {
+  multiarch/'memory.c': {
+    'gdb_test': ['--test', files('../gdbstub/memory.py')]
+  },
+  multiarch/'interrupt.c': {
+    'gdb_test': ['--test', files('../gdbstub/interrupt.py')]
+  },
+  multiarch/'hello.c': {
+    'test_name': 'untimely-packet',
+    'gdb_test': ['--gdb-args',  '-ex \'set debug remote 1\'',
+                 '--output', '/dev/stdout',
+                 '--stderr', '/dev/stdout'],
+    'wrapper': [prog_run_and_check_forbidden_output,
+                'Packet instead of Ack, ignoring it']
+  }
+}
+
+tests += {
+  multiarch/'memory.c': {
+    'test_name': 'registers',
+    'gdb_test': ['--test', files('../gdbstub/registers.py')]
+  }
+}
+
+tests += {
+  multiarch/'memory.c': {
+    'test_name': 'memory-access',
+    'plugin_test': {
+      'plugin': 'mem',
+      'args': ['region-summary=true'],
+    },
+    'wrapper': [prog_check_plugin_output,
+                find_program('./validate-memory-counts.py')]
+  }
+}
+
+tcg_tests += {
+  'multiarch-softmmu': {
+    'tests': tests
+  }
+}
-- 
2.43.0

[PATCH 26/27] tests/tcg/aarch64: add multiarch system tests

[Pierrick Bouvier]

We can now add multiarch system tests. Because of our policy to not
repeat cflags, we must make sure we only add those once per executable.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/aarch64/system/meson.build | 31 ++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/tests/tcg/aarch64/system/meson.build b/tests/tcg/aarch64/system/meson.build
index a1edf1b8442..2a613cca52d 100644
--- a/tests/tcg/aarch64/system/meson.build
+++ b/tests/tcg/aarch64/system/meson.build
@@ -17,6 +17,32 @@ qemu_base_args = ['-display', 'none',
                   '-kernel']
 qemu_def_args = ['-M', 'virt', '-cpu', 'max', qemu_base_args]
 
+# Multi arch tests - add cflags only once per src
+multi_src = []
+foreach t: tcg_tests['multiarch-softmmu']['tests']
+  foreach src, setup: t
+    new_cflags = cflags
+    if fs.stem(src) == 'memory'
+      new_cflags += '-DCHECK_UNALIGNED=1'
+    endif
+    if src not in multi_src
+      setup += {'cflags': [new_cflags]}
+      multi_src += src
+    endif
+    tests += {src: setup + {'qemu_args': qemu_def_args}}
+  endforeach
+endforeach
+
+if cc_has_feat['sve']
+  tests += {
+    '../../multiarch/system/memory.c': {
+      'exe_name': 'memory-sve',
+      'cflags': [cflags, feat_cflags['sve'], '-DCHECK_UNALIGNED=1', '-O3'],
+      'qemu_args': qemu_def_args,
+    },
+  }
+endif
+
 tests += {
   'asid2.c': {'cflags': cflags, 'qemu_args': qemu_def_args},
   'feat-xs.c': {'cflags': cflags, 'qemu_args': qemu_def_args},
@@ -30,6 +56,11 @@ tests += {
                   qemu_def_args],
     'wrapper': [prog_run_with_input, 'X']
   },
+  '../../multiarch/system/memory.c': {
+    'test_name': 'memory-record-replay',
+    'qemu_args': qemu_def_args,
+    'wrapper': [prog_record_replay],
+  },
   'vtimer.c': {
     'cflags': cflags,
     'qemu_args': ['-M', 'virt,virtualization=on,gic-version=2',
-- 
2.43.0

[PATCH 27/27] tests/tcg/meson.build: add generic plugin tests

[Pierrick Bouvier]

We finally run every multiarch test (user & system) with every plugin we
have. We can implement another strategy easily, the point here is just
to show how we can do this without impacting test definitions files.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
 tests/tcg/meson.build | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 9ec33739884..e88c018fb6e 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -70,6 +70,42 @@ subdir('plugins')
 # multiarch comes second, so it can be referenced by other arch after
 subdir('multiarch')
 
+# Generic Plugin tests - add plugin for every user/system multiarch test
+plugin_tests_exclude = ['patch']
+foreach target: ['multiarch-linux-user', 'multiarch-softmmu']
+  plugin_tests = []
+  tests = tcg_tests[target]['tests']
+
+  foreach plugin_name, _: test_plugins
+    if plugin_name in plugin_tests_exclude
+      continue
+    endif
+
+    foreach t: tests
+      foreach src, setup: t
+        if 'plugin_test' in setup or 'gdb_test' in setup
+          continue
+        endif
+
+        # copy existing setup and remove existing cflags
+        new_setup = {}
+        foreach key, val: setup
+          if key == 'cflags'
+            continue
+          endif
+          new_setup += {key: val}
+        endforeach
+
+        new_setup += {'plugin_test': {'plugin': plugin_name}}
+        plugin_tests += {src: new_setup}
+      endforeach
+    endforeach
+  endforeach
+
+  # replace list of tests with old and new ones
+  tcg_tests += {target: {'tests': tests + plugin_tests}}
+endforeach
+
 # Now let's go through all architectures
 subdir('aarch64')
 
-- 
2.43.0

Re: [PATCH 00/27] tests/tcg: run tests with meson

[Pierrick Bouvier]

For reference:

On 6/9/2026 2:47 PM, Pierrick Bouvier wrote:
> It implements all requirements listed on original thread [1], except cross
> container support, which can be easily added on top once we agree on the base.
> [1] https://lore.kernel.org/qemu-devel/87wlwfsay4.fsf@draig.linaro.org/
> 

>>   - support building tests with containers

to be implemented in future version, but nothing blocking.

>>   - handle all existing tests
>>     - per-arch *-user and *-softmmu
>>       - including post test validation checks (conditional-diff-out,
validate-memory-counts.py, etc...)

Done by adding the 'wrapper' concept.

>>       - per-target overrides (some targets are broken for some tests)

Each target includes what it wants, so it's free to skip/customize any test.

>>       - custom QEMU_OPTS for some tests

Done by adding 'qemu_args'.

>>       - including sub-arch tests (e.g. x86_64 includes i386, mips64el
includes mips64 etc...)

Each target includes what it wants, including sub-arch if needed.

>>     - multiarch tests for all *-user targets
>>     - multiarch tests for all *-softmmu targets with boot.S
>>     - gdbstub tests

Done by adding 'gdb_test'.

>>       - don't forget gdb feature probing for arch and arch features
(e.g. GDB_HAS_SME_TILES)

Done by comparing gdb version as needed from meson.

>>     - record/replay tests

Done by adding a record_replay wrapper.

>>     - tcg plugin tests

Done by adding 'plugin_test'.

>>       - modulate plugins over tests to avoid combinatorial explosion

Not implemented at the moment, but we can use any heuristic to limit the
PxT expansion.

>>       - filter some plugins from general testing (e.g. patch)
>>       - filter some plugins based on mode (e.g. syscall)
>>       - specific plugins with specific tests (e.g. patch, memory)

Each plugin test is explicit, so we are free to implement any policy.

>>     - support build variations

Done by adding 'cflags' and ensuring we can't create an accidental
duplication.

>>       - compiler feature probing to gate tests

Done directly in meson by testing compiler.

>>       - custom cflags for some tests

Done by adding 'cflags'.

>>       - alternate cflag builds for some tests (e.g.
sha512-[vector|sse|mvx|...])

Done by specifying a different 'cflags', and an alternate executable
name with 'exe_name'. We ensure duplication is explicit.

>>       - build scripts (e.g. test-mmx.py)

Not needed in aarch64 at the moment. For this use case, we'll generate a
header using a custom target, which will be a dependency of final test.

Regards,
Pierrick

Re: [PATCH 21/27] tests/tcg/aarch64: add user tests

[Richard Henderson]

On 6/9/26 14:47, Pierrick Bouvier wrote:
> +++ b/tests/tcg/aarch64/meson.build
> @@ -0,0 +1,213 @@
> +cc = find_program('aarch64-linux-gnu-gcc', required : false)
> +
> +if not cc.found()
> +  subdir_done()
> +endif

The primary thing we lose here is the ability to use containers for the cross-compile.

We detect cross-compilers in configure.  We currently write them to 
build/tests/tcg/${target}/config-target.mak.  It's not very meson to parse a file like 
that, but I imagine we could use scripts or meson variables somehow.


r~

Re: [PATCH 01/27] tests/tcg/multiarch/system/memory.c: remove unused variable

[Richard Henderson]

On 6/9/26 14:47, Pierrick Bouvier wrote:
> Caught this since by compiling this file with -Wall, -Werror.
> 
> ../tests/tcg/aarch64/system/../../multiarch/system/memory.c: In function ‘init_test_data_s8’:
> ../tests/tcg/aarch64/system/../../multiarch/system/memory.c:97:18: error: unused variable ‘bottom’ [-Werror=unused-variable]
>     97 |     uint8_t top, bottom, *ptr = &test_data[0];
>        |                  ^~~~~~
> ../tests/tcg/aarch64/system/../../multiarch/system/memory.c:97:13: error: unused variable ‘top’ [-Werror=unused-variable]
>     97 |     uint8_t top, bottom, *ptr = &test_data[0];
>        |             ^~~
> ../tests/tcg/aarch64/system/../../multiarch/system/memory.c: In function ‘read_test_data_s8’:
> ../tests/tcg/aarch64/system/../../multiarch/system/memory.c:384:14: error: unused variable ‘ok’ [-Werror=unused-variable]
>    384 |         bool ok;
>        |
> 
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>


Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 02/27] tests/tcg/multiarch/plugin/check-plugin-output.sh: take test output as input

[Richard Henderson]

On 6/9/26 14:47, Pierrick Bouvier wrote:
> Instead of executing test, just read it's output from a file. This
> allows to standardize how to call check plugins scripts.
> 
> Signed-off-by: Pierrick Bouvier<pierrick.bouvier@oss.qualcomm.com>
> ---
>   tests/tcg/multiarch/Makefile.target           |  3 +--
>   .../multiarch/plugin/check-plugin-output.sh   | 22 ++++++-------------
>   2 files changed, 8 insertions(+), 17 deletions(-)

Could use "done < $test_output" instead of cat, but

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 03/27] tests/tcg/multiarch/plugin: rename check-plugin-output to regex-compare

[Richard Henderson]

On 6/9/26 14:47, Pierrick Bouvier wrote:
> We'll reuse this name later in the series, so change existing script.
> 
> Signed-off-by: Pierrick Bouvier<pierrick.bouvier@oss.qualcomm.com>
> ---
>   tests/tcg/multiarch/Makefile.target                             | 2 +-
>   .../plugin/{check-plugin-output.sh => regex-compare.sh}         | 0
>   2 files changed, 1 insertion(+), 1 deletion(-)
>   rename tests/tcg/multiarch/plugin/{check-plugin-output.sh => regex-compare.sh} (100%)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 21/27] tests/tcg/aarch64: add user tests

[Pierrick Bouvier]

On 6/9/2026 5:18 PM, Richard Henderson wrote:
> On 6/9/26 14:47, Pierrick Bouvier wrote:
>> +++ b/tests/tcg/aarch64/meson.build
>> @@ -0,0 +1,213 @@
>> +cc = find_program('aarch64-linux-gnu-gcc', required : false)
>> +
>> +if not cc.found()
>> +  subdir_done()
>> +endif
> 
> The primary thing we lose here is the ability to use containers for the
> cross-compile.
>

It's definitely on the requirements list, but as listed in the cover
letter, out of the scope for current series.
> We detect cross-compilers in configure.  We currently write them to
> build/tests/tcg/${target}/config-target.mak.  It's not very meson to
> parse a file like that, but I imagine we could use scripts or meson
> variables somehow.
>

We can rewrite this mechanic in meson directly, similar to what we do
for gdb in this series. find_program from a list of known names, else
fallback to our cross containers which have a known and fixed setup.

The "build_container_and_run_compiler" logic will be in an external
script, appearing as a cross compiler itself, so meson role will be
limited to find a given program and test some flags.

> 
> r~

Re: [PATCH 16/27] tests/tcg/meson.build: introduce expected_output

[Pierrick Bouvier]

On 6/9/2026 2:47 PM, Pierrick Bouvier wrote:
> A test can declare it expects a specific output on stdout. In this case,
> we compare it to a reference file. We implement this instead of relying
> on a wrapper because we need the flexibility to resolve name of
> reference file per target.
> 
> For instance, multiarch float tests expect different ref files for each
> target, which is something we can't express with a test wrapper.
> 
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
>  tests/tcg/meson.build             |  9 ++++++++-
>  tests/tcg/scripts/run_and_diff.sh | 12 ++++++++++++
>  2 files changed, 20 insertions(+), 1 deletion(-)
>  create mode 100755 tests/tcg/scripts/run_and_diff.sh
> 
> diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
> index 5380a360830..3d2c0d6583b 100644
> --- a/tests/tcg/meson.build
> +++ b/tests/tcg/meson.build
> @@ -7,6 +7,7 @@ if config_host.has_key('GDB')
>  endif
>  gdb = find_program(gdb_progs, required: false)
>  prog_gdb_test = find_program('../guest-debug/run-test.py')
> +prog_run_and_diff = find_program('./scripts/run_and_diff.sh')
>  
>  tcg_tests = {}
>  # tcg_tests is a dictionary with following structure:
> @@ -24,6 +25,7 @@ tcg_tests = {}
>  #           'test_name': ['provide an alternative test name'],
>  #           'cflags': ['cflags to compile test', ...],
>  #           'qemu_args': ['qemu command line flags', ...],
> +#           'expected_output': 'file_to_compare_output',
>  #           'gdb_test': ['gdb test args'],
>  #           'env_var': ['VAR=value', ...],
>  #           'wrapper': [program_to_launch_test, 'args'...],
> @@ -84,7 +86,7 @@ foreach target, plan: tcg_tests
>      foreach src, setup: t
>        # return a clear error if user mispell a setup entry
>        foreach key, _ : setup
> -        if key not in ['cflags', 'qemu_args', 'gdb_test',
> +        if key not in ['cflags', 'qemu_args', 'expected_output', 'gdb_test',
>                         'exe_name', 'test_name', 'env_var', 'plugin_test',
>                         'wrapper']
>            error('unknown tcg setup entry \'' + key + '\' for test ' + src)
> @@ -197,6 +199,11 @@ foreach target, plan: tcg_tests
>                ]
>        endif
>  
> +      if 'expected_output' in setup
> +        expected = files(folder / setup['expected_output'])
> +        cmd = [prog_run_and_diff, expected, cmd]
> +      endif
> +
>        if 'wrapper' in setup
>          cmd = [setup['wrapper'], cmd]
>        endif
> diff --git a/tests/tcg/scripts/run_and_diff.sh b/tests/tcg/scripts/run_and_diff.sh
> new file mode 100755
> index 00000000000..d1e9243c49d
> --- /dev/null
> +++ b/tests/tcg/scripts/run_and_diff.sh
> @@ -0,0 +1,12 @@
> +#!/usr/bin/env bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +set -euo pipefail
> +
> +if [ $# -lt 2 ]; then
> +    echo "run_and_diff: expected_output_file cmd [args]..." 1>&2
> +    exit 1
> +fi
> +expected="$1";shift
> +set -x
> +diff <("$@") $expected

Fixed this line, since it hides return code of program.
Write to a temporary file, then diff.

Re: [PATCH 04/27] tests/tcg: introduce meson.build

[Pierrick Bouvier]

On 6/9/2026 2:47 PM, Pierrick Bouvier wrote:
> Every target declare its tests, and add them in global tcg_tests
> dictionary, indexed by target name. See comment for full description.
> 
> We make sure that if a user repeats a test several time it won't result
> in duplicated executables or tests. In next commits, we'll introduce
> what's needed to deal with that.
> 
> A test is compiled from a single source file (and later, a single set of
> compile flags). We make sure dependencies are correctly handled by
> generating a depfile.
> 
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
>  tests/meson.build     |  2 +-
>  tests/tcg/meson.build | 94 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 95 insertions(+), 1 deletion(-)
>  create mode 100644 tests/tcg/meson.build
> 
> diff --git a/tests/meson.build b/tests/meson.build
> index 9ba04bbedd3..5877d2f60b1 100644
> --- a/tests/meson.build
> +++ b/tests/meson.build
> @@ -74,7 +74,7 @@ subdir('decode')
>  
>  if 'CONFIG_TCG' in config_all_accel
>    subdir('fp')
> -  subdir('tcg/plugins')
> +  subdir('tcg')
>  endif
>  
>  subdir('audio')
> diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
> new file mode 100644
> index 00000000000..72d1a9f8986
> --- /dev/null
> +++ b/tests/tcg/meson.build
> @@ -0,0 +1,94 @@
> +env = find_program('env')
> +
> +tcg_tests = {}
> +# tcg_tests is a dictionary with following structure:
> +#
> +# {
> +#   'name_of_target': {
> +#     'cc': cross_compiler,
> +#     'folder': 'folder_for_test_src_files',
> +#     'qemu': emulators['qemu-...'],
> +#     'tests': [
> +#       {
> +#         'src_file': {
> +#         }
> +#       },
> +#       ...
> +#     ]
> +#   }
> +# }
> +#
> +# Every test executable, is built only once.
> +# Tests for a given src use the same executable by default, and their definition
> +# is guaranteed to be unique also.
> +# Default name is derived from src.
> +
> +# plugins come first, as we need to build the list
> +subdir('plugins')
> +
> +# Finally, we can create all test executables and test targets
> +foreach target, plan: tcg_tests
> +  # Detect duplicated executables/tests, and report an error to force user to
> +  # choose how to deal with it.
> +  built_tests = {}
> +  added_tests = {}
> +
> +  # return a clear error if user mispell a target entry
> +  foreach key, _ : plan
> +    if key not in ['cc', 'folder', 'qemu', 'tests']
> +      error('unknown tcg test plan entry \'' + key + '\' for target ' + target)
> +    endif
> +  endforeach
> +
> +  cc = plan['cc']
> +  folder = plan['folder']
> +  qemu = plan['qemu']
> +  tests = plan['tests']
> +
> +  foreach t : tests
> +    foreach src, setup: t
> +      # meson '/' operator drops left operand if right is an absolute path
> +      src = folder / src
> +      file = files(src)
> +      test = fs.name(file)

dead variable, removed.

> +      exe_name = fs.stem(src)
> +
> +      exe_name = target + '-' + exe_name
> +      test_name = exe_name
> +
> +      if test_name in added_tests
> +        error('test ' + test_name + ' was already added: ' +
> +              'specify a different \'test_name\'')
> +      endif
> +
> +      # build executable if needed
> +      if exe_name not in built_tests
> +        exe = custom_target(exe_name,
> +                            input: file,
> +                            command: [cc, '@INPUT@',
> +                                      '-o', '@OUTPUT@', '-static',
> +                                      '-MMD', '-MF', '@DEPFILE@',
> +                                      '-Wall', '-Werror',
> +                                      '-O0', '-g', '-fno-strict-aliasing',
> +                                      ],
> +                            depfile: exe_name + '.d',
> +                            output: exe_name + '.test',
> +                            build_by_default: false)
> +        built_tests += {exe_name: exe}
> +      endif
> +
> +      depends = []
> +
> +      exe = built_tests[exe_name]
> +      cmd = [qemu, exe]
> +
> +      # some tests expect QEMU env var to be set
> +      cmd = ['QEMU=' + qemu.full_path(), cmd]
> +
> +      test(test_name, env, args: cmd,
> +           depends: depends,
> +           suite: ['tcg', 'tcg-' + target])
> +      added_tests += {test_name: true}
> +    endforeach
> +  endforeach
> +endforeach

Re: [PATCH 05/27] tests/tcg/meson.build: introduce exe_name

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>We add possibility to duplicate a test executable for a given source
>file. This is needed when different cflags need to be used (like
>vectorized vs non-vectorized for instance, like sve tests in aarch64).
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---
> tests/tcg/meson.build | 17 +++++++++++++++--
> 1 file changed, 15 insertions(+), 2 deletions(-)
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index 72d1a9f8986..2c6d85c586a 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -11,6 +11,7 @@ tcg_tests = {}
> #     'tests': [
> #       {
> #         'src_file': {
>+#           'exe_name': ['provide an alternative binary name'],
> #         }
> #       },
> #       ...
>@@ -18,10 +19,10 @@ tcg_tests = {}
> #   }
> # }
> #
>-# Every test executable, is built only once.
>+# Every test executable, identified by 'exe_name' is built only once.
> # Tests for a given src use the same executable by default, and their definition
> # is guaranteed to be unique also.
>-# Default name is derived from src.
>+# Default name is derived from src if 'exe_name' is omitted.
> 
> # plugins come first, as we need to build the list
> subdir('plugins')
>@@ -47,11 +48,23 @@ foreach target, plan: tcg_tests
> 
>   foreach t : tests
>     foreach src, setup: t
>+      # return a clear error if user mispell a setup entry

Minor typo:

  If user misspells*

Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>

>+      foreach key, _ : setup
>+        if key not in [
>+                       'exe_name',
>+                       ]
>+          error('unknown tcg setup entry \'' + key + '\' for test ' + src)
>+        endif
>+      endforeach
>+
>       # meson '/' operator drops left operand if right is an absolute path
>       src = folder / src
>       file = files(src)
>       test = fs.name(file)
>       exe_name = fs.stem(src)
>+      if 'exe_name' in setup
>+        exe_name = setup['exe_name']
>+      endif
> 
>       exe_name = target + '-' + exe_name
>       test_name = exe_name
>-- 
>2.43.0
>

Re: [PATCH 06/27] tests/tcg/meson.build: introduce test_name

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>We add possibility to duplicate a test for a given src. This allows to
>declare several tests that still share the same executable, thus saving
>space and compile time.
>
>A side note to mention that meson perfectly supports declaring several
>tests with the same name. However, we prevent that to force dev to
>clarify what is the intent of each individual test.
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---
> tests/tcg/meson.build | 14 ++++++++++----
> 1 file changed, 10 insertions(+), 4 deletions(-)
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index 2c6d85c586a..bb0184732fd 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -12,6 +12,7 @@ tcg_tests = {}
> #       {
> #         'src_file': {
> #           'exe_name': ['provide an alternative binary name'],
>+#           'test_name': ['provide an alternative test name'],
> #         }
> #       },
> #       ...
>@@ -21,8 +22,8 @@ tcg_tests = {}
> #
> # Every test executable, identified by 'exe_name' is built only once.
> # Tests for a given src use the same executable by default, and their definition
>-# is guaranteed to be unique also.
>-# Default name is derived from src if 'exe_name' is omitted.
>+# is guaranteed to be unique also. They can be duplicated by setting 'test_name'.

While it's clear in the patch description, it might be a good idea to 
explain that "duplication" means using the same executable here, for 
clarity, if you respin this series.

>+# Default name is derived from src if 'exe_name' and 'test_name' are omitted.
> 
> # plugins come first, as we need to build the list
> subdir('plugins')
>@@ -51,7 +52,7 @@ foreach target, plan: tcg_tests
>       # return a clear error if user mispell a setup entry
>       foreach key, _ : setup
>         if key not in [
>-                       'exe_name',
>+                       'exe_name', 'test_name',
>                        ]
>           error('unknown tcg setup entry \'' + key + '\' for test ' + src)


Unrelated to this patch, but you could refactor the ['exe_name', 
'test_name'] etc key set outside and print the list of valid keys in the 
error message as well for UX.

>         endif
>@@ -66,8 +67,13 @@ foreach target, plan: tcg_tests
>         exe_name = setup['exe_name']
>       endif
> 
>-      exe_name = target + '-' + exe_name
>       test_name = exe_name
>+      if 'test_name' in setup
>+        test_name = setup['test_name']
>+      endif
>+
>+      exe_name = target + '-' + exe_name
>+      test_name = target + '-' + test_name
> 
>       if test_name in added_tests
>         error('test ' + test_name + ' was already added: ' +
>-- 
>2.43.0
>


LGTM (feel free to ignore the comments)

Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>

Re: [PATCH 07/27] tests/tcg/meson.build: introduce cflags

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>We add possibility to declare cflags for a given test. Since we want to
>preserve the uniqueness of test binaries, we prevent repeating those
>cflags later, for an existing binary.
>
>This ensures one executable == one set of cflags, and if dev needs
>duplication, they can use a custom 'exe_name' to override this.
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---

Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>

> tests/tcg/meson.build | 17 ++++++++++++++---
> 1 file changed, 14 insertions(+), 3 deletions(-)
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index bb0184732fd..ad56fdf4ab5 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -13,6 +13,7 @@ tcg_tests = {}
> #         'src_file': {
> #           'exe_name': ['provide an alternative binary name'],
> #           'test_name': ['provide an alternative test name'],
>+#           'cflags': ['cflags to compile test', ...],
> #         }
> #       },
> #       ...
>@@ -20,7 +21,8 @@ tcg_tests = {}
> #   }
> # }
> #
>-# Every test executable, identified by 'exe_name' is built only once.
>+# Every test executable, identified by 'exe_name' is built only once, and we
>+# ensure cflags are specified only once, on first declaration.
> # Tests for a given src use the same executable by default, and their definition
> # is guaranteed to be unique also. They can be duplicated by setting 'test_name'.
> # Default name is derived from src if 'exe_name' and 'test_name' are omitted.
>@@ -51,7 +53,7 @@ foreach target, plan: tcg_tests
>     foreach src, setup: t
>       # return a clear error if user mispell a setup entry
>       foreach key, _ : setup
>-        if key not in [
>+        if key not in ['cflags',
>                        'exe_name', 'test_name',
>                        ]
>           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
>@@ -80,6 +82,15 @@ foreach target, plan: tcg_tests
>               'specify a different \'test_name\'')
>       endif
> 
>+      cflags = []
>+      if 'cflags' in setup
>+        if exe_name in built_tests
>+          error('test ' + exe_name + ' was already built with cflags: ' +
>+                'remove cflags or set \'exe_name\' (if cflags are different)')
>+        endif
>+        cflags = setup['cflags']
>+      endif
>+
>       # build executable if needed
>       if exe_name not in built_tests
>         exe = custom_target(exe_name,
>@@ -89,7 +100,7 @@ foreach target, plan: tcg_tests
>                                       '-MMD', '-MF', '@DEPFILE@',
>                                       '-Wall', '-Werror',
>                                       '-O0', '-g', '-fno-strict-aliasing',
>-                                      ],
>+                                      cflags],
>                             depfile: exe_name + '.d',
>                             output: exe_name + '.test',
>                             build_by_default: false)
>-- 
>2.43.0
>

Re: [PATCH 08/27] tests/tcg/meson.build: introduce qemu_args

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>We add possibility to use custom arguments when calling qemu for a test.
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---

Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>

> tests/tcg/meson.build | 9 +++++++--
> 1 file changed, 7 insertions(+), 2 deletions(-)
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index ad56fdf4ab5..28a7e529c7e 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -14,6 +14,7 @@ tcg_tests = {}
> #           'exe_name': ['provide an alternative binary name'],
> #           'test_name': ['provide an alternative test name'],
> #           'cflags': ['cflags to compile test', ...],
>+#           'qemu_args': ['qemu command line flags', ...],
> #         }
> #       },
> #       ...
>@@ -53,7 +54,7 @@ foreach target, plan: tcg_tests
>     foreach src, setup: t
>       # return a clear error if user mispell a setup entry
>       foreach key, _ : setup
>-        if key not in ['cflags',
>+        if key not in ['cflags', 'qemu_args',
>                        'exe_name', 'test_name',
>                        ]
>           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
>@@ -108,9 +109,13 @@ foreach target, plan: tcg_tests
>       endif
> 
>       depends = []
>+      qemu_args = []
>+      if 'qemu_args' in setup
>+        qemu_args = setup['qemu_args']
>+      endif
> 
>       exe = built_tests[exe_name]
>-      cmd = [qemu, exe]
>+      cmd = [qemu, qemu_args, exe]
> 
>       # some tests expect QEMU env var to be set
>       cmd = ['QEMU=' + qemu.full_path(), cmd]
>-- 
>2.43.0
>

Re: [PATCH 09/27] tests/tcg/meson.build: introduce env_var

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>We add possibility to set an environment variable when running a test.
>An example of a test requiring this is
>tests/tcg/multiarch/linux/linux-sigrtminmax.c.
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---

Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>

> tests/tcg/meson.build | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index 28a7e529c7e..b6a375e9eee 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -15,6 +15,7 @@ tcg_tests = {}
> #           'test_name': ['provide an alternative test name'],
> #           'cflags': ['cflags to compile test', ...],
> #           'qemu_args': ['qemu command line flags', ...],
>+#           'env_var': ['VAR=value', ...],
> #         }
> #       },
> #       ...
>@@ -55,7 +56,7 @@ foreach target, plan: tcg_tests
>       # return a clear error if user mispell a setup entry
>       foreach key, _ : setup
>         if key not in ['cflags', 'qemu_args',
>-                       'exe_name', 'test_name',
>+                       'exe_name', 'test_name', 'env_var',
>                        ]
>           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
>         endif
>@@ -120,6 +121,10 @@ foreach target, plan: tcg_tests
>       # some tests expect QEMU env var to be set
>       cmd = ['QEMU=' + qemu.full_path(), cmd]
> 
>+      if 'env_var' in setup
>+        cmd = [setup['env_var'], cmd]
>+      endif
>+
>       test(test_name, env, args: cmd,
>            depends: depends,
>            suite: ['tcg', 'tcg-' + target])
>-- 
>2.43.0
>

Re: [PATCH 10/27] tests/tcg/plugins: build list of test_plugins

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>We previously had list of test_plugins sources, but not the target
>themselves. Create this list, so tests can then reference a plugin by
>its name from there.
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---

Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>

> tests/tcg/meson.build         |  1 +
> tests/tcg/plugins/meson.build | 11 +++++++----
> 2 files changed, 8 insertions(+), 4 deletions(-)
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index b6a375e9eee..5cd80f3c030 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -30,6 +30,7 @@ tcg_tests = {}
> # Default name is derived from src if 'exe_name' and 'test_name' are omitted.
> 
> # plugins come first, as we need to build the list
>+test_plugins = {}
> subdir('plugins')
> 
> # Finally, we can create all test executables and test targets
>diff --git a/tests/tcg/plugins/meson.build b/tests/tcg/plugins/meson.build
>index d7f8f0ae0ad..2abc317d1e7 100644
>--- a/tests/tcg/plugins/meson.build
>+++ b/tests/tcg/plugins/meson.build
>@@ -1,4 +1,4 @@
>-test_plugins = [
>+test_plugins_src = [
> 'bb.c',
> 'discons.c',
> 'empty.c',
>@@ -14,9 +14,12 @@ test_plugins = [
> 
> t = []
> if get_option('plugins')
>-  foreach i : test_plugins
>-    t += shared_module(fs.stem(i), files(i),
>-                       dependencies: plugins_deps)
>+  foreach i : test_plugins_src
>+    name = fs.stem(i)
>+    plugin = shared_module(name, files(i),
>+                           dependencies: plugins_deps)
>+    t += plugin
>+    test_plugins += {name: plugin}
>   endforeach
> endif
> if t.length() > 0
>-- 
>2.43.0
>

Re: [PATCH 11/27] tests/tcg/meson.build: introduce plugin_test

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>We add possibility to declare a test that is based on a plugin.
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---
> tests/tcg/meson.build | 40 +++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 39 insertions(+), 1 deletion(-)
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index 5cd80f3c030..a47f1ada779 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -16,6 +16,10 @@ tcg_tests = {}
> #           'cflags': ['cflags to compile test', ...],
> #           'qemu_args': ['qemu command line flags', ...],
> #           'env_var': ['VAR=value', ...],
>+#           'plugin_test': {
>+#             'plugin': 'plugin_name',
>+#             'args': ['plugin_args', ...],
>+#           },
> #         }
> #       },
> #       ...
>@@ -57,12 +61,25 @@ foreach target, plan: tcg_tests
>       # return a clear error if user mispell a setup entry
>       foreach key, _ : setup
>         if key not in ['cflags', 'qemu_args',
>-                       'exe_name', 'test_name', 'env_var',
>+                       'exe_name', 'test_name', 'env_var', 'plugin_test',
>                        ]
>           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
>         endif
>       endforeach
> 
>+      # return a clear error if user mispell a plugin entry

misspells :D haha

>+      if 'plugin_test' in setup
>+        foreach key, _ : setup['plugin_test']
>+          if key not in ['plugin', 'args']
>+            error('unknown tcg plugin entry \'' + key + '\' for test ' + src)
>+          endif
>+        endforeach
>+      endif
>+
>+      if 'plugin_test' in setup and not get_option('plugins')
>+        continue
>+      endif
>+
>       # meson '/' operator drops left operand if right is an absolute path
>       src = folder / src
>       file = files(src)
>@@ -76,6 +93,12 @@ foreach target, plan: tcg_tests
>       if 'test_name' in setup
>         test_name = setup['test_name']
>       endif
>+      if 'plugin_test' in setup
>+        # remove lib prefix
>+        plugin = test_plugins[setup['plugin_test']['plugin']]
>+        plugin_name = fs.stem(plugin).substring(3)

Why skip the first 3 chars? Maybe add a comment like you do elsewhere 
for unclear lines (suggestion).

Regardless of my comments:

Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>

>+        test_name = 'plugin-' + plugin_name + '-with-' + test_name
>+      endif
> 
>       exe_name = target + '-' + exe_name
>       test_name = target + '-' + test_name
>@@ -116,6 +139,21 @@ foreach target, plan: tcg_tests
>         qemu_args = setup['qemu_args']
>       endif
> 
>+      if 'plugin_test' in setup
>+        plugin_setup = setup['plugin_test']
>+        plugin = test_plugins[plugin_setup['plugin']]
>+        plugin_args = []
>+        if 'args' in plugin_setup
>+          # add empty arg to generate an additional comma
>+          plugin_args = ['', plugin_setup['args']]
>+        endif
>+        plugin_args = ','.join(plugin_args)
>+
>+        # since we turn plugin into a string, add dependency explicitly
>+        depends += plugin
>+        qemu_args = ['-plugin', plugin.full_path() + plugin_args, qemu_args]
>+      endif
>+
>       exe = built_tests[exe_name]
>       cmd = [qemu, qemu_args, exe]
> 
>-- 
>2.43.0
>

Re: [PATCH 12/27] tests/tcg/meson.build: test gdb support and introduce gdb_arch

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>A given target can now declare which gdb architecture it requires.
>Also, we add the code to check gdb capabilities, and version.
>Tests can use gdb_version.version_compare('>=X.Y') to test this.
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---

Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>

> tests/tcg/meson.build | 22 +++++++++++++++++++++-
> 1 file changed, 21 insertions(+), 1 deletion(-)
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index a47f1ada779..1496cba460c 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -1,5 +1,11 @@
> env = find_program('env')
> 
>+gdb_progs = ['gdb-multiarch', 'gdb']
>+if config_host.has_key('GDB')
>+  gdb_progs = [config_host['GDB'], gdb_progs]
>+endif
>+gdb = find_program(gdb_progs, required: false)
>+
> tcg_tests = {}
> # tcg_tests is a dictionary with following structure:
> #
>@@ -7,6 +13,7 @@ tcg_tests = {}
> #   'name_of_target': {
> #     'cc': cross_compiler,
> #     'folder': 'folder_for_test_src_files',
>+#     'gdb_arch': 'gdb architecture requirement',
> #     'qemu': emulators['qemu-...'],
> #     'tests': [
> #       {
>@@ -33,6 +40,18 @@ tcg_tests = {}
> # is guaranteed to be unique also. They can be duplicated by setting 'test_name'.
> # Default name is derived from src if 'exe_name' and 'test_name' are omitted.
> 
>+gdb_arch_supported = {}
>+gdb_version = '0'
>+if gdb.found()
>+  gdb_version = gdb.version()
>+  gdb_arch = run_command(python, files('../../scripts/probe-gdb-support.py'),
>+                           gdb,
>+                           check: true)
>+  foreach arch: gdb_arch.stdout().split()
>+    gdb_arch_supported += {arch: true}
>+  endforeach
>+endif
>+
> # plugins come first, as we need to build the list
> test_plugins = {}
> subdir('plugins')
>@@ -46,13 +65,14 @@ foreach target, plan: tcg_tests
> 
>   # return a clear error if user mispell a target entry
>   foreach key, _ : plan
>-    if key not in ['cc', 'folder', 'qemu', 'tests']
>+    if key not in ['cc', 'folder', 'gdb_arch', 'qemu', 'tests']
>       error('unknown tcg test plan entry \'' + key + '\' for target ' + target)
>     endif
>   endforeach
> 
>   cc = plan['cc']
>   folder = plan['folder']
>+  gdb_arch = plan['gdb_arch']
>   qemu = plan['qemu']
>   tests = plan['tests']
> 
>-- 
>2.43.0
>

Re: [PATCH 13/27] tests/tcg/meson.build: introduce gdb_test

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>We can now declare a test requiring gdb interaction, based on
>tests/guest-debug/run-test.py.
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---


Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>

> tests/tcg/meson.build | 22 ++++++++++++++++++++--
> 1 file changed, 20 insertions(+), 2 deletions(-)
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index 1496cba460c..d0080eb0f9f 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -5,6 +5,7 @@ if config_host.has_key('GDB')
>   gdb_progs = [config_host['GDB'], gdb_progs]
> endif
> gdb = find_program(gdb_progs, required: false)
>+prog_gdb_test = find_program('../guest-debug/run-test.py')
> 
> tcg_tests = {}
> # tcg_tests is a dictionary with following structure:
>@@ -22,6 +23,7 @@ tcg_tests = {}
> #           'test_name': ['provide an alternative test name'],
> #           'cflags': ['cflags to compile test', ...],
> #           'qemu_args': ['qemu command line flags', ...],
>+#           'gdb_test': ['gdb test args'],
> #           'env_var': ['VAR=value', ...],
> #           'plugin_test': {
> #             'plugin': 'plugin_name',
>@@ -80,7 +82,7 @@ foreach target, plan: tcg_tests
>     foreach src, setup: t
>       # return a clear error if user mispell a setup entry
>       foreach key, _ : setup
>-        if key not in ['cflags', 'qemu_args',
>+        if key not in ['cflags', 'qemu_args', 'gdb_test',
>                        'exe_name', 'test_name', 'env_var', 'plugin_test',
>                        ]
>           error('unknown tcg setup entry \'' + key + '\' for test ' + src)
>@@ -96,6 +98,10 @@ foreach target, plan: tcg_tests
>         endforeach
>       endif
> 
>+      if 'gdb_test' in setup and gdb_arch not in gdb_arch_supported
>+        continue
>+      endif
>+
>       if 'plugin_test' in setup and not get_option('plugins')
>         continue
>       endif
>@@ -113,7 +119,9 @@ foreach target, plan: tcg_tests
>       if 'test_name' in setup
>         test_name = setup['test_name']
>       endif
>-      if 'plugin_test' in setup
>+      if 'gdb_test' in setup
>+        test_name = 'gdb-' + test_name
>+      elif 'plugin_test' in setup
>         # remove lib prefix
>         plugin = test_plugins[setup['plugin_test']['plugin']]
>         plugin_name = fs.stem(plugin).substring(3)
>@@ -177,6 +185,16 @@ foreach target, plan: tcg_tests
>       exe = built_tests[exe_name]
>       cmd = [qemu, qemu_args, exe]
> 
>+      if 'gdb_test' in setup
>+        cmd = [prog_gdb_test,
>+               '--gdb', gdb,
>+               '--qemu', qemu,
>+               '--qargs', ' '.join(qemu_args),
>+               '--bin', exe,
>+               setup['gdb_test'],
>+              ]
>+      endif
>+
>       # some tests expect QEMU env var to be set
>       cmd = ['QEMU=' + qemu.full_path(), cmd]
> 
>-- 
>2.43.0
>

Re: [PATCH 16/27] tests/tcg/meson.build: introduce expected_output

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 07:30, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>On 6/9/2026 2:47 PM, Pierrick Bouvier wrote:
>> A test can declare it expects a specific output on stdout. In this case,
>> we compare it to a reference file. We implement this instead of relying
>> on a wrapper because we need the flexibility to resolve name of
>> reference file per target.
>> 
>> For instance, multiarch float tests expect different ref files for each
>> target, which is something we can't express with a test wrapper.
>> 
>> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>> ---
>>  tests/tcg/meson.build             |  9 ++++++++-
>>  tests/tcg/scripts/run_and_diff.sh | 12 ++++++++++++
>>  2 files changed, 20 insertions(+), 1 deletion(-)
>>  create mode 100755 tests/tcg/scripts/run_and_diff.sh
>> 
>> diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>> index 5380a360830..3d2c0d6583b 100644
>> --- a/tests/tcg/meson.build
>> +++ b/tests/tcg/meson.build
>> @@ -7,6 +7,7 @@ if config_host.has_key('GDB')
>>  endif
>>  gdb = find_program(gdb_progs, required: false)
>>  prog_gdb_test = find_program('../guest-debug/run-test.py')
>> +prog_run_and_diff = find_program('./scripts/run_and_diff.sh')
>>  
>>  tcg_tests = {}
>>  # tcg_tests is a dictionary with following structure:
>> @@ -24,6 +25,7 @@ tcg_tests = {}
>>  #           'test_name': ['provide an alternative test name'],
>>  #           'cflags': ['cflags to compile test', ...],
>>  #           'qemu_args': ['qemu command line flags', ...],
>> +#           'expected_output': 'file_to_compare_output',
>>  #           'gdb_test': ['gdb test args'],
>>  #           'env_var': ['VAR=value', ...],
>>  #           'wrapper': [program_to_launch_test, 'args'...],
>> @@ -84,7 +86,7 @@ foreach target, plan: tcg_tests
>>      foreach src, setup: t
>>        # return a clear error if user mispell a setup entry
>>        foreach key, _ : setup
>> -        if key not in ['cflags', 'qemu_args', 'gdb_test',
>> +        if key not in ['cflags', 'qemu_args', 'expected_output', 'gdb_test',
>>                         'exe_name', 'test_name', 'env_var', 'plugin_test',
>>                         'wrapper']
>>            error('unknown tcg setup entry \'' + key + '\' for test ' + src)
>> @@ -197,6 +199,11 @@ foreach target, plan: tcg_tests
>>                ]
>>        endif
>>  
>> +      if 'expected_output' in setup
>> +        expected = files(folder / setup['expected_output'])
>> +        cmd = [prog_run_and_diff, expected, cmd]
>> +      endif
>> +
>>        if 'wrapper' in setup
>>          cmd = [setup['wrapper'], cmd]
>>        endif
>> diff --git a/tests/tcg/scripts/run_and_diff.sh b/tests/tcg/scripts/run_and_diff.sh
>> new file mode 100755
>> index 00000000000..d1e9243c49d
>> --- /dev/null
>> +++ b/tests/tcg/scripts/run_and_diff.sh
>> @@ -0,0 +1,12 @@
>> +#!/usr/bin/env bash
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +set -euo pipefail
>> +
>> +if [ $# -lt 2 ]; then
>> +    echo "run_and_diff: expected_output_file cmd [args]..." 1>&2
>> +    exit 1
>> +fi
>> +expected="$1";shift
>> +set -x
>> +diff <("$@") $expected
>
>Fixed this line, since it hides return code of program.
>Write to a temporary file, then diff.

Also, consider if using #!/bin/sh is possible, since it's universally 
portable.

Re: [PATCH 18/27] tests/tcg/meson.build: add wrapper run_with_input

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>Semihosting console tests require console interaction, and were flagged
>as manual because of this. This wrapper automates interaction, by
>sending a specific string on stdin.
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---

Fyi there is also expect(1) https://packages.debian.org/trixie/expect 
but is kind of arcane.

> tests/tcg/meson.build               |  1 +
> tests/tcg/scripts/run_with_input.sh | 12 ++++++++++++
> 2 files changed, 13 insertions(+)
> create mode 100755 tests/tcg/scripts/run_with_input.sh
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index f379449e582..7201a7809e1 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -8,6 +8,7 @@ endif
> gdb = find_program(gdb_progs, required: false)
> prog_gdb_test = find_program('../guest-debug/run-test.py')
> prog_run_and_diff = find_program('./scripts/run_and_diff.sh')
>+prog_run_with_input = find_program('./scripts/run_with_input.sh')
> prog_run_and_check_forbidden_output = find_program(
>   './scripts/run_and_check_forbidden_output.sh'
> )
>diff --git a/tests/tcg/scripts/run_with_input.sh b/tests/tcg/scripts/run_with_input.sh
>new file mode 100755
>index 00000000000..3d280547c3a
>--- /dev/null
>+++ b/tests/tcg/scripts/run_with_input.sh
>@@ -0,0 +1,12 @@
>+#!/usr/bin/env bash
>+# SPDX-License-Identifier: GPL-2.0-or-later
>+
>+set -euo pipefail
>+
>+if [ $# -lt 2 ]; then
>+    echo "run_with_input: input_string cmd [args]..." 1>&2
>+    exit 1
>+fi
>+input="$1";shift
>+set -x
>+echo "$input" | "$@"
>-- 
>2.43.0
>

Re: [PATCH 20/27] tests/tcg/meson.build: add wrapper check_plugin_output

[Manos Pitsidianakis]

On Wed, 10 Jun 2026 00:47, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> wrote:
>This wrapper runs a qemu command, record command output and plugin
>output, and forward both to a check script.
>We use this pattern in several plugin tests (for instance, with
>validate-memory-counts.py).
>
>Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>---
> tests/tcg/meson.build                    |  1 +
> tests/tcg/scripts/check_plugin_output.sh | 19 +++++++++++++++++++
> 2 files changed, 20 insertions(+)
> create mode 100755 tests/tcg/scripts/check_plugin_output.sh
>
>diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>index f0efd46ee3e..6e46c121fda 100644
>--- a/tests/tcg/meson.build
>+++ b/tests/tcg/meson.build
>@@ -6,6 +6,7 @@ if config_host.has_key('GDB')
>   gdb_progs = [config_host['GDB'], gdb_progs]
> endif
> gdb = find_program(gdb_progs, required: false)
>+prog_check_plugin_output = find_program('./scripts/check_plugin_output.sh')
> prog_gdb_test = find_program('../guest-debug/run-test.py')
> prog_record_replay = find_program('./scripts/record_replay.sh')
> prog_run_and_diff = find_program('./scripts/run_and_diff.sh')
>diff --git a/tests/tcg/scripts/check_plugin_output.sh b/tests/tcg/scripts/check_plugin_output.sh
>new file mode 100755
>index 00000000000..4d12e23ad60
>--- /dev/null
>+++ b/tests/tcg/scripts/check_plugin_output.sh
>@@ -0,0 +1,19 @@
>+#!/usr/bin/env bash
>+# SPDX-License-Identifier: GPL-2.0-or-later
>+
>+set -euo pipefail
>+
>+if [ $# -lt 3 ]; then
>+    echo "check_plugin_output: check_cmd qemu_bin [args]..." 1>&2
>+    exit 1
>+fi
>+check_cmd="$1"; shift
>+qemu="$1"; shift
>+tmp=$(mktemp -d)
>+trap "rm -rf $tmp" EXIT
>+
>+set -x
>+
>+$qemu -d plugin -D $tmp/plugin "$@" |& tee $tmp/output
>+cat $tmp/plugin
>+$check_cmd $tmp/output $tmp/plugin
>-- 

Could we write a single python script for all the shell script wrappers 
and use a different kind of wrapper depending on its command line 
flags/subcommand we invoke it with? 

Maybe this could be better? WDYT?

Re: [PATCH 15/27] tests/tcg/meson.build: introduce wrapper

[Philippe Mathieu-Daudé]

On 9/6/26 23:47, Pierrick Bouvier wrote:
> We add possibility to wrap a test command through a given script. This
> can be used to provide any kind of ad-hoc check for specific tests.
> 
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
>   tests/tcg/meson.build | 7 ++++++-
>   1 file changed, 6 insertions(+), 1 deletion(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>

Re: [PATCH 14/27] tests/tcg/meson.build: add default flag for testing compiler support

[Philippe Mathieu-Daudé]

On 9/6/26 23:47, Pierrick Bouvier wrote:
> This global variable can be used as cflags when checking if a compiler
> supports specific flags.
> 
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
>   tests/tcg/meson.build | 1 +
>   1 file changed, 1 insertion(+)

Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>

Re: [PATCH 08/27] tests/tcg/meson.build: introduce qemu_args

[Philippe Mathieu-Daudé]

On 9/6/26 23:47, Pierrick Bouvier wrote:
> We add possibility to use custom arguments when calling qemu for a test.
> 
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
>   tests/tcg/meson.build | 9 +++++++--
>   1 file changed, 7 insertions(+), 2 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>

Re: [PATCH 21/27] tests/tcg/aarch64: add user tests

[Paolo Bonzini]

On 6/10/26 02:18, Richard Henderson wrote:
> On 6/9/26 14:47, Pierrick Bouvier wrote:
>> +++ b/tests/tcg/aarch64/meson.build
>> @@ -0,0 +1,213 @@
>> +cc = find_program('aarch64-linux-gnu-gcc', required : false)
>> +
>> +if not cc.found()
>> +  subdir_done()
>> +endif
> 
> The primary thing we lose here is the ability to use containers for the 
> cross-compile.
> 
> We detect cross-compilers in configure.  We currently write them to 
> build/tests/tcg/${target}/config-target.mak.  It's not very meson to 
> parse a file like that, but I imagine we could use scripts or meson 
> variables somehow.

That's not a problem, we can write them in [binaries] and find_program() 
will pick it up automatically.  I'm a bit more worried about having to 
reinvent cross compilation altogether, I'll reply elsewhere.

Paolo

Re: [PATCH 00/27] tests/tcg: run tests with meson

[Paolo Bonzini]

On 6/9/26 23:47, Pierrick Bouvier wrote:
> This series is an experiment to replace TCG tests makefiles with meson, with the
> goal of having something more readable and maintainable in the long term.
> Also it closes the gap that tcg tests had with the rest of tests, by integrating
> them with meson directly.
> 
> It covers all existing tests for aarch64 architecture, including
> user/system/multiarch-user/multiarch-system. I'll carry the effort to port all
> other architectures once we agree on a base. We agreed with Alex to not merge
> anything before all arch are covered.
> 
> Series first declare the global infrastructure, then add aarch64 and multiarch
> tests, and finally plugin tests.
> 
> For review, I would suggest to start with aarch64 patches (end of the series) to
> get an idea of what they will look like, before the implementation itself. You
> can also compare that to existing Makefiles, they follow the same order.
> 
> It implements all requirements listed on original thread [1], except cross
> container support, which can be easily added on top once we agree on the base.
> [1] https://lore.kernel.org/qemu-devel/87wlwfsay4.fsf@draig.linaro.org/

> We also have for free:
> - correct and complete dependencies for any test, including:
>    scripts, c.inc, headers, reference files and binaries/plugins.
> - catch test declaration issues at configure time vs test time.
> - proper data types for variables instead of string expansion with make.
> - we could now build tests binaries by default with 'all' target.
> - hopefully, better readability to your taste.

Yes there is absolutely a readability improvement!  At most, one could 
say that it's not a very high bar. :)

Having to reinvent cross compilation (cc_has_feat, custom_targets for 
compilation) is not great and the only part I'm not super convinced 
about; I wonder what it would be like to make TCG tests their own 
standalone meson projects.  While the Makefiles are awful I do like 
being able to do "make -C tests/tcg/x86_64-softmmu", and in that respect 
this would be a half step backwards (and a big step forwards in many 
other respects, mind).

I can help with the integration of the individual test suites into 
tests/tcg/Makefile.include.

Paolo

> Main ideas for the current design are:
> - make arch files as simple and explicit as possible. If it conflicts,
>    always pick explicit, as long as we don't expose meson details.
>    remove any kind of "skip, override, filter" logic.
> - never expose custom_target and meson details in arch files.
> - catch any spelling mistake on test or any dependency.
> - keep complexity contained in tests/tcg/meson.build.
> 
> Usage:
> 
> ```
> make check-tcg
> make check-tcg-aarch64-linux-user
> make check-tcg-aarch64-softmmu
> 
> from build folder:
> ./pyvenv/bin/meson test --list --suite tcg
> ninja clean
> ./pyvenv/bin/meson test aarch64-softmmu-asid2 --verbose
> ```
> 
> Pierrick Bouvier (27):
>    tests/tcg/multiarch/system/memory.c: remove unused variable
>    tests/tcg/multiarch/plugin/check-plugin-output.sh: take test output as
>      input
>    tests/tcg/multiarch/plugin: rename check-plugin-output to
>      regex-compare
>    tests/tcg: introduce meson.build
>    tests/tcg/meson.build: introduce exe_name
>    tests/tcg/meson.build: introduce test_name
>    tests/tcg/meson.build: introduce cflags
>    tests/tcg/meson.build: introduce qemu_args
>    tests/tcg/meson.build: introduce env_var
>    tests/tcg/plugins: build list of test_plugins
>    tests/tcg/meson.build: introduce plugin_test
>    tests/tcg/meson.build: test gdb support and introduce gdb_arch
>    tests/tcg/meson.build: introduce gdb_test
>    tests/tcg/meson.build: add default flag for testing compiler support
>    tests/tcg/meson.build: introduce wrapper
>    tests/tcg/meson.build: introduce expected_output
>    tests/tcg/meson.build: add wrapper run_and_check_forbidden_output
>    tests/tcg/meson.build: add wrapper run_with_input
>    tests/tcg/meson.build: add wrapper record_replay
>    tests/tcg/meson.build: add wrapper check_plugin_output
>    tests/tcg/aarch64: add user tests
>    tests/tcg/aarch64: add system tests
>    tests/tcg/multiarch: declare user tests
>    tests/tcg/aarch64: add multiarch user tests
>    tests/tcg/multiarch: declare system tests
>    tests/tcg/aarch64: add multiarch system tests
>    tests/tcg/meson.build: add generic plugin tests
> 
>   tests/meson.build                             |   2 +-
>   tests/tcg/aarch64/meson.build                 | 248 ++++++++++++++++
>   tests/tcg/aarch64/system/meson.build          | 100 +++++++
>   tests/tcg/meson.build                         | 276 ++++++++++++++++++
>   tests/tcg/multiarch/Makefile.target           |   3 +-
>   tests/tcg/multiarch/meson.build               | 149 ++++++++++
>   .../multiarch/plugin/check-plugin-output.sh   |  36 ---
>   tests/tcg/multiarch/plugin/regex-compare.sh   |  28 ++
>   tests/tcg/multiarch/sha1.ref                  |   1 +
>   tests/tcg/multiarch/system/memory.c           |   3 +-
>   tests/tcg/multiarch/system/meson.build        |  53 ++++
>   tests/tcg/plugins/meson.build                 |  11 +-
>   tests/tcg/scripts/check_plugin_output.sh      |  19 ++
>   tests/tcg/scripts/record_replay.sh            |  16 +
>   .../scripts/run_and_check_forbidden_output.sh |  19 ++
>   tests/tcg/scripts/run_and_diff.sh             |  12 +
>   tests/tcg/scripts/run_with_input.sh           |  12 +
>   17 files changed, 943 insertions(+), 45 deletions(-)
>   create mode 100644 tests/tcg/aarch64/meson.build
>   create mode 100644 tests/tcg/aarch64/system/meson.build
>   create mode 100644 tests/tcg/meson.build
>   create mode 100644 tests/tcg/multiarch/meson.build
>   delete mode 100755 tests/tcg/multiarch/plugin/check-plugin-output.sh
>   create mode 100755 tests/tcg/multiarch/plugin/regex-compare.sh
>   create mode 100644 tests/tcg/multiarch/sha1.ref
>   create mode 100644 tests/tcg/multiarch/system/meson.build
>   create mode 100755 tests/tcg/scripts/check_plugin_output.sh
>   create mode 100755 tests/tcg/scripts/record_replay.sh
>   create mode 100755 tests/tcg/scripts/run_and_check_forbidden_output.sh
>   create mode 100755 tests/tcg/scripts/run_and_diff.sh
>   create mode 100755 tests/tcg/scripts/run_with_input.sh
>