[PATCH v1 0/5] run unit tests in the CI

[PATCH v1 0/5] run unit tests in the CI

[Edwin Török]

Currently the unit tests aren't run in the CI during the build.
Introduce a 'make check' rule that can run them all (in parallel if `-j`
is specified).

To make reading the CI output easier make's `-O` flag is used (which
buffers output per target, making it obvious which command failed).

TBD: what is the actual minimum version of binutils required by the
existing tests, because Ubuntu 16.04 with binutil 2.26.1 failed.
That version of Ubuntu is EoL in a few days, so it is dropped from the CI build step.
The actual containers and the xilinx-arm64 jobs are kept for now
(although these should be updated to use bionic).

A followup patch series will rearrange the unit test make rules,
so it is useful to test that all the unit tests worked both prior and
following that change on all supported platforms in the CI.

For convenience this series is also available as a git repository:
https://gitlab.com/xen-project/people/edwintorok/xen/-/tree/private/edvint/run-ci?ref_type=heads

Edwin Török (5):
  README: update minimum make to 4.1
  CI: drop Ubuntu 16.04
  tools/test: introduce a 'make check' rule
  automation/scripts/build: run unit tests in the CI
  automation/scripts/build: group command output from parallel jobs

 Makefile                        |  4 ++++
 README                          |  2 +-
 automation/gitlab-ci/build.yaml | 10 ----------
 automation/scripts/build        | 10 ++++++----
 tools/Makefile                  |  4 ++++
 tools/Rules.mk                  |  2 +-
 tools/tests/Makefile            | 20 +++++++++++++++++++-
 7 files changed, 35 insertions(+), 17 deletions(-)

-- 
2.47.3

[PATCH v1 1/5] README: update minimum make to 4.1

[Edwin Török]

Using .DEFAULT_GOAL requires at least GNU make 3.81.
It was suggested to update make to match ~2015 era gcc/binutils, hence
4.1.

Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Suggested-by: Jan Beulich <jbeulich@suse.com>
Link: https://lore.kernel.org/xen-devel/c0ba57bb-0d86-4209-b019-daf8328b5205@suse.com/
---
 README | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/README b/README
index 889a4ea906..af1f539d29 100644
--- a/README
+++ b/README
@@ -35,7 +35,7 @@ Second, there are a number of prerequisites for building a Xen source
 release. Make sure you have all the following installed, either by
 visiting the project webpage or installing a pre-built package
 provided by your OS distributor:
-    * GNU Make v3.80 or later
+    * GNU Make v4.1 or later
     * C compiler and linker:
       - For x86:
         - GCC 5.1 or later
-- 
2.47.3

Re: [PATCH v1 1/5] README: update minimum make to 4.1

[Jan Beulich]

On 30.03.2026 18:17, Edwin Török wrote:
> Using .DEFAULT_GOAL requires at least GNU make 3.81.
> It was suggested to update make to match ~2015 era gcc/binutils, hence
> 4.1.
> 
> Signed-off-by: Edwin Török <edwin.torok@citrix.com>
> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Link: https://lore.kernel.org/xen-devel/c0ba57bb-0d86-4209-b019-daf8328b5205@suse.com/

Unless we're specifically meaning to use 4.1 features, personally I'd prefer
the lower 4.0 (which is what I have on my main dev workstation).

Also, nit: Tags in chronological order, please.

Jan

> --- a/README
> +++ b/README
> @@ -35,7 +35,7 @@ Second, there are a number of prerequisites for building a Xen source
>  release. Make sure you have all the following installed, either by
>  visiting the project webpage or installing a pre-built package
>  provided by your OS distributor:
> -    * GNU Make v3.80 or later
> +    * GNU Make v4.1 or later
>      * C compiler and linker:
>        - For x86:
>          - GCC 5.1 or later

Re: [PATCH v1 1/5] README: update minimum make to 4.1

[Edwin Torok]

> On 31 Mar 2026, at 07:14, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 30.03.2026 18:17, Edwin Török wrote:
>> Using .DEFAULT_GOAL requires at least GNU make 3.81.
>> It was suggested to update make to match ~2015 era gcc/binutils, hence
>> 4.1.
>> 
>> Signed-off-by: Edwin Török <edwin.torok@citrix.com>
>> Suggested-by: Jan Beulich <jbeulich@suse.com>
>> Link: https://lore.kernel.org/xen-devel/c0ba57bb-0d86-4209-b019-daf8328b5205@suse.com/
> 
> Unless we're specifically meaning to use 4.1 features, personally I'd prefer
> the lower 4.0


OK.

> (which is what I have on my main dev workstation).

Which OS/distribution are you using?
It’d be best to add that to the CI, such that there is at least one configuration testing the minimum versions listed in the README.
(Or at least the intersection of the versions of build tools in the CI should match the ones in the README)

Best regards,
—Edwin

> 
> Also, nit: Tags in chronological order, please.
> 
> Jan
> 
>> --- a/README
>> +++ b/README
>> @@ -35,7 +35,7 @@ Second, there are a number of prerequisites for building a Xen source
>> release. Make sure you have all the following installed, either by
>> visiting the project webpage or installing a pre-built package
>> provided by your OS distributor:
>> -    * GNU Make v3.80 or later
>> +    * GNU Make v4.1 or later
>>     * C compiler and linker:
>>       - For x86:
>>         - GCC 5.1 or later
> 

Re: [PATCH v1 1/5] README: update minimum make to 4.1

[Jan Beulich]

On 31.03.2026 10:15, Edwin Torok wrote:
>> On 31 Mar 2026, at 07:14, Jan Beulich <jbeulich@suse.com> wrote:
>> On 30.03.2026 18:17, Edwin Török wrote:
>>> Using .DEFAULT_GOAL requires at least GNU make 3.81.
>>> It was suggested to update make to match ~2015 era gcc/binutils, hence
>>> 4.1.
>>>
>>> Signed-off-by: Edwin Török <edwin.torok@citrix.com>
>>> Suggested-by: Jan Beulich <jbeulich@suse.com>
>>> Link: https://lore.kernel.org/xen-devel/c0ba57bb-0d86-4209-b019-daf8328b5205@suse.com/
>>
>> Unless we're specifically meaning to use 4.1 features, personally I'd prefer
>> the lower 4.0
> 
> 
> OK.
> 
>> (which is what I have on my main dev workstation).
> 
> Which OS/distribution are you using?

On that particular system it's SLES12 SP5 (in LTSS).

Jan

Re: [PATCH v1 1/5] README: update minimum make to 4.1

[Edwin Torok]

> On 31 Mar 2026, at 09:24, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 31.03.2026 10:15, Edwin Torok wrote:
>>> On 31 Mar 2026, at 07:14, Jan Beulich <jbeulich@suse.com> wrote:
>>> On 30.03.2026 18:17, Edwin Török wrote:
>>>> Using .DEFAULT_GOAL requires at least GNU make 3.81.
>>>> It was suggested to update make to match ~2015 era gcc/binutils, hence
>>>> 4.1.
>>>> 
>>>> Signed-off-by: Edwin Török <edwin.torok@citrix.com>
>>>> Suggested-by: Jan Beulich <jbeulich@suse.com>
>>>> Link: https://lore.kernel.org/xen-devel/c0ba57bb-0d86-4209-b019-daf8328b5205@suse.com/
>>> 
>>> Unless we're specifically meaning to use 4.1 features, personally I'd prefer
>>> the lower 4.0
>> 
>> 
>> OK.
>> 
>>> (which is what I have on my main dev workstation).
>> 
>> Which OS/distribution are you using?
> 
> On that particular system it's SLES12 SP5 (in LTSS).
> 
> Jan

Might be difficult to get that exact version into the CI (https://registry.suse.com/repositories/suse-ltss-sle12-5-sles12sp5 says it requires a login), unless someone with access to that version sets up a Gitlab runner.
One way to approximate it might be to use https://hub.docker.com/layers/opensuse/leap/42.3/images/sha256-84d4bef2c4657b7bfb48233aa84f40e2303af0607de06562cf4afc496ec18b59, which should approximate an older SLES12 SP3.
If it builds and passes tests there, then I assume it should also build on SLES12 SP5 (except for toolchain bugs).
At least the version of make seems close enough (4.0-7.15), and that way contributors can test themselves whether their patches build with the minimum requirements specified in the README.

Best regards,
--Edwin

[PATCH v1 2/5] CI: drop Ubuntu 16.04

[Edwin Török]

Ubuntu 16.04 is EoL on 2026-04-02.
It fails to build the emulator tests, probably due to a binutils that is too old:

```
gcc -m32 -march=i686 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wno-unused-but-set-variable -Wno-unused-local-typedefs -Werror -O2 -fomit-frame-pointer -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -mno-tls-direct-seg-refs -fno-pie -fno-exceptions -fno-asynchronous-unwind-tables -ffreestanding -nostdinc -I/builds/xen-project/people/edwintorok/xen/tools/tests/x86_emulator/../../../tools/firmware/include -fno-stack-protector -g0 -D_16 -mpclmul -mssse3 -mpclmul -ffixed-xmm0 -Os -DVEC_SIZE=16 -c ssse3-pclmul.c
/tmp/cchhD6n5.s: Assembler messages:
/tmp/cchhD6n5.s:202: Error: junk at end of line, first unrecognized character is `{'
/tmp/cchhD6n5.s:203: Error: junk at end of line, first unrecognized character is `{'
/tmp/cchhD6n5.s:205: Error: junk at end of line, first unrecognized character is `{'
```

Same test passes on Ubuntu 18.04.

Note: the minimum version of binutils might have to be updated.
Ubuntu 16.04 had version 2.26.1, which satisfies the >= 2.25 requirement
in the README, and yet it failed as shown above.

Signed-off-by: Edwin Török <edwin.torok@citrix.com>
---
 automation/gitlab-ci/build.yaml | 10 ----------
 1 file changed, 10 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index b69bad9202..41914cf898 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -615,16 +615,6 @@ fedora-41-x86_64-gcc-debug:
   variables:
     CONTAINER: fedora:41-x86_64
 
-ubuntu-16.04-x86_64-gcc:
-  extends: .gcc-x86-64-build
-  variables:
-    CONTAINER: ubuntu:16.04-x86_64
-
-ubuntu-16.04-x86_64-gcc-debug:
-  extends: .gcc-x86-64-build-debug
-  variables:
-    CONTAINER: ubuntu:16.04-x86_64
-
 ubuntu-18.04-x86_64-gcc:
   extends: .gcc-x86-64-build
   variables:
-- 
2.47.3

Re: [PATCH v1 2/5] CI: drop Ubuntu 16.04

[Jan Beulich]

On 30.03.2026 18:17, Edwin Török wrote:
> Ubuntu 16.04 is EoL on 2026-04-02.

It going EOL very soon is a good reason; it causing ...

> It fails to build the emulator tests, probably due to a binutils that is too old:
> 
> ```
> gcc -m32 -march=i686 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wno-unused-but-set-variable -Wno-unused-local-typedefs -Werror -O2 -fomit-frame-pointer -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -mno-tls-direct-seg-refs -fno-pie -fno-exceptions -fno-asynchronous-unwind-tables -ffreestanding -nostdinc -I/builds/xen-project/people/edwintorok/xen/tools/tests/x86_emulator/../../../tools/firmware/include -fno-stack-protector -g0 -D_16 -mpclmul -mssse3 -mpclmul -ffixed-xmm0 -Os -DVEC_SIZE=16 -c ssse3-pclmul.c
> /tmp/cchhD6n5.s: Assembler messages:
> /tmp/cchhD6n5.s:202: Error: junk at end of line, first unrecognized character is `{'
> /tmp/cchhD6n5.s:203: Error: junk at end of line, first unrecognized character is `{'
> /tmp/cchhD6n5.s:205: Error: junk at end of line, first unrecognized character is `{'
> ```

... build issues in the test blobs isn't. The harness is specifically able to
cope with blob build failures. Another thing would be if test_x86_emulator.c
failed to build (but see below).

Is the above representative output anyway (i.e. is this not perhaps interleaved
output from a parallel build)? ssse3-pclmul.c, built with -mssse3 -mpclmul (i.e.
no AVX512 options), shouldn't really involve `{'. Furthermore we specifically
have a check in the Makefile, skipping building altogether when gcc and/or gas
are too old.

> Same test passes on Ubuntu 18.04.

Hard to believe that there wouldn't be at least some failures. Perhaps said
check prevents the attempt to build the harness there?

> Note: the minimum version of binutils might have to be updated.
> Ubuntu 16.04 had version 2.26.1, which satisfies the >= 2.25 requirement
> in the README, and yet it failed as shown above.

The harness is special, as said. Imo we shouldn't be updating the requirements
just for it. If anything, the mentioned gcc/gas check may need updating. {evex},
for example, requires gas 2.29 (i.e. gcc6 time frame). As does VPCMPESTRIQ. I
specifically have a local patch to make it possible to build the harness with
pre-gcc7 (I don't have any gcc6 anywhere, to that boundary may be off by 1). I
didn't think this hackery would be acceptable upstream.

Jan

Re: [PATCH v1 2/5] CI: drop Ubuntu 16.04

[Edwin Torok]

> On 31 Mar 2026, at 07:58, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 30.03.2026 18:17, Edwin Török wrote:
>> Ubuntu 16.04 is EoL on 2026-04-02.
> 
> It going EOL very soon is a good reason; it causing ...
> 
>> It fails to build the emulator tests, probably due to a binutils that is too old:
>> 
>> ```
>> gcc -m32 -march=i686 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wno-unused-but-set-variable -Wno-unused-local-typedefs -Werror -O2 -fomit-frame-pointer -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -mno-tls-direct-seg-refs -fno-pie -fno-exceptions -fno-asynchronous-unwind-tables -ffreestanding -nostdinc -I/builds/xen-project/people/edwintorok/xen/tools/tests/x86_emulator/../../../tools/firmware/include -fno-stack-protector -g0 -D_16 -mpclmul -mssse3 -mpclmul -ffixed-xmm0 -Os -DVEC_SIZE=16 -c ssse3-pclmul.c
>> /tmp/cchhD6n5.s: Assembler messages:
>> /tmp/cchhD6n5.s:202: Error: junk at end of line, first unrecognized character is `{'
>> /tmp/cchhD6n5.s:203: Error: junk at end of line, first unrecognized character is `{'
>> /tmp/cchhD6n5.s:205: Error: junk at end of line, first unrecognized character is `{'
>> ```
> 
> ... build issues in the test blobs isn't. The harness is specifically able to
> cope with blob build failures. Another thing would be if test_x86_emulator.c
> failed to build (but see below).

The whole build pipeline failed, maybe I extracted the wrong part of the error message.
https://gitlab.com/xen-project/people/edwintorok/xen/-/jobs/13661296490

Unfortunately the output is truncated:
```
Job's log exceeded limit of 4194304 bytes.
Job execution will continue but no more output will be collected.
```

It looks like GCC accepted the mavx512dq flag, but then binutils failed to assemble the output?
Although as you say another way to avoid that would be to fix the gas version check,
more on that below.

> 
> Is the above representative output anyway (i.e. is this not perhaps interleaved
> output from a parallel build)? ssse3-pclmul.c, built with -mssse3 -mpclmul (i.e.
> no AVX512 options), shouldn't really involve `{'. Furthermore we specifically
> have a check in the Makefile, skipping building altogether when gcc and/or gas
> are too old.
> 
>> Same test passes on Ubuntu 18.04.
> 
> Hard to believe that there wouldn't be at least some failures. Perhaps said
> check prevents the attempt to build the harness there?

The passing build logs are at https://gitlab.com/xen-project/people/edwintorok/xen/-/jobs/13661296494/viewer
There are failures about unrecognised compiler flags, which are indeed ignored:
```
gcc: error: unrecognized command line option '-mavx512fp16'; did you mean '-mavx512f'?
testcase.mk:16: recipe for target 'avx512fp16.bin' failed
make[7]: *** [avx512fp16.bin] Error 1
```

> 
>> Note: the minimum version of binutils might have to be updated.
>> Ubuntu 16.04 had version 2.26.1, which satisfies the >= 2.25 requirement
>> in the README, and yet it failed as shown above.
> 
> The harness is special, as said. Imo we shouldn't be updating the requirements
> just for it. If anything, the mentioned gcc/gas check may need updating. {evex},
> for example, requires gas 2.29 (i.e. gcc6 time frame).

I think the problem might be that the checks aren’t done for the ‘run’ target (which is what I’m attempting to use in the CI):
```
# Suppress building by default of the harness if the compiler can't deal
# with some of the extensions used. Don't alter the "run" target dependencies
# though, as this target needs to be specified manually, and things may work
# partially even with older compilers.

TARGET-y := $(TARGET)

ifneq ($(filter-out run% clean% distclean,$(MAKECMDGOALS)),)
```

The simplest solution here would be to remove ‘run%’ from the filter rules.
If it is useful I can introduce an alias force-run, that can be used to manually try to run anyway (although that attempt may in the end fail, as with Ubuntu 16.04).
What do you think?

Best regards,
—Edwin
> As does VPCMPESTRIQ. I
> specifically have a local patch to make it possible to build the harness with
> pre-gcc7 (I don't have any gcc6 anywhere, to that boundary may be off by 1). I
> didn't think this hackery would be acceptable upstream.
> 
> Jan

Re: [PATCH v1 2/5] CI: drop Ubuntu 16.04

[Jan Beulich]

On 31.03.2026 10:29, Edwin Torok wrote:
> 
> 
>> On 31 Mar 2026, at 07:58, Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 30.03.2026 18:17, Edwin Török wrote:
>>> Ubuntu 16.04 is EoL on 2026-04-02.
>>
>> It going EOL very soon is a good reason; it causing ...
>>
>>> It fails to build the emulator tests, probably due to a binutils that is too old:
>>>
>>> ```
>>> gcc -m32 -march=i686 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wno-unused-but-set-variable -Wno-unused-local-typedefs -Werror -O2 -fomit-frame-pointer -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -mno-tls-direct-seg-refs -fno-pie -fno-exceptions -fno-asynchronous-unwind-tables -ffreestanding -nostdinc -I/builds/xen-project/people/edwintorok/xen/tools/tests/x86_emulator/../../../tools/firmware/include -fno-stack-protector -g0 -D_16 -mpclmul -mssse3 -mpclmul -ffixed-xmm0 -Os -DVEC_SIZE=16 -c ssse3-pclmul.c
>>> /tmp/cchhD6n5.s: Assembler messages:
>>> /tmp/cchhD6n5.s:202: Error: junk at end of line, first unrecognized character is `{'
>>> /tmp/cchhD6n5.s:203: Error: junk at end of line, first unrecognized character is `{'
>>> /tmp/cchhD6n5.s:205: Error: junk at end of line, first unrecognized character is `{'
>>> ```
>>
>> ... build issues in the test blobs isn't. The harness is specifically able to
>> cope with blob build failures. Another thing would be if test_x86_emulator.c
>> failed to build (but see below).
> 
> The whole build pipeline failed, maybe I extracted the wrong part of the error message.
> https://gitlab.com/xen-project/people/edwintorok/xen/-/jobs/13661296490
> 
> Unfortunately the output is truncated:
> ```
> Job's log exceeded limit of 4194304 bytes.
> Job execution will continue but no more output will be collected.
> ```

With that we of course can't find out what's wrong.

> It looks like GCC accepted the mavx512dq flag, but then binutils failed to assemble the output?

But -mavx512dq would be passed only when building blobs, which ...

> Although as you say another way to avoid that would be to fix the gas version check,
> more on that below.
> 
>>
>> Is the above representative output anyway (i.e. is this not perhaps interleaved
>> output from a parallel build)? ssse3-pclmul.c, built with -mssse3 -mpclmul (i.e.
>> no AVX512 options), shouldn't really involve `{'. Furthermore we specifically
>> have a check in the Makefile, skipping building altogether when gcc and/or gas
>> are too old.
>>
>>> Same test passes on Ubuntu 18.04.
>>
>> Hard to believe that there wouldn't be at least some failures. Perhaps said
>> check prevents the attempt to build the harness there?
> 
> The passing build logs are at https://gitlab.com/xen-project/people/edwintorok/xen/-/jobs/13661296494/viewer
> There are failures about unrecognised compiler flags, which are indeed ignored:
> ```
> gcc: error: unrecognized command line option '-mavx512fp16'; did you mean '-mavx512f'?
> testcase.mk:16: recipe for target 'avx512fp16.bin' failed
> make[7]: *** [avx512fp16.bin] Error 1
> ```

... is tolerated to fail.

>>> Note: the minimum version of binutils might have to be updated.
>>> Ubuntu 16.04 had version 2.26.1, which satisfies the >= 2.25 requirement
>>> in the README, and yet it failed as shown above.
>>
>> The harness is special, as said. Imo we shouldn't be updating the requirements
>> just for it. If anything, the mentioned gcc/gas check may need updating. {evex},
>> for example, requires gas 2.29 (i.e. gcc6 time frame).
> 
> I think the problem might be that the checks aren’t done for the ‘run’ target (which is what I’m attempting to use in the CI):
> ```
> # Suppress building by default of the harness if the compiler can't deal
> # with some of the extensions used. Don't alter the "run" target dependencies
> # though, as this target needs to be specified manually, and things may work
> # partially even with older compilers.
> 
> TARGET-y := $(TARGET)
> 
> ifneq ($(filter-out run% clean% distclean,$(MAKECMDGOALS)),)
> ```
> 
> The simplest solution here would be to remove ‘run%’ from the filter rules.

No, as said elsewhere, run% being different is intentional.

> If it is useful I can introduce an alias force-run, that can be used to manually try to run anyway (although that attempt may in the end fail, as with Ubuntu 16.04).
> What do you think?

Maybe. Or maybe we should have "check" targets alongside the "run" ones,
being aliases of one another perhaps for everything but the emulator
harness?

Jan

[PATCH v1 3/5] tools/test: introduce a 'make check' rule

[Edwin Török]

It runs all unit tests (those that do not require xenctrl) in parallel.
The for loop fragment from tools/Rules.mk is not used, because that
serializes make invocations in subdirectories (tested by inserting a
'sleep 5' into the 'run' rules).

There might be dependencies between directories in tools/,
but all tests are independent and can be built and run in parallel.

Signed-off-by: Edwin Török <edwin.torok@citrix.com>
---
 Makefile             |  4 ++++
 tools/Makefile       |  4 ++++
 tools/Rules.mk       |  2 +-
 tools/tests/Makefile | 20 +++++++++++++++++++-
 4 files changed, 28 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 67b71ac3d4..ca7ec58c99 100644
--- a/Makefile
+++ b/Makefile
@@ -50,6 +50,10 @@ install: $(TARGS_INSTALL)
 .PHONY: build
 build: $(TARGS_BUILD)
 
+.PHONY: check
+check:
+	$(MAKE) -C tools check
+
 .PHONY: build-xen
 build-xen:
 	$(MAKE) -C xen build
diff --git a/tools/Makefile b/tools/Makefile
index 6ecf7c0da8..7f6c29965d 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -55,6 +55,10 @@ endif
 .PHONY: build all
 build all: subdirs-all
 
+.PHONY: check
+check:
+	$(MAKE) -C tests check
+
 .PHONY: install
 install:
 	$(INSTALL_DIR) -m 700 $(DESTDIR)$(XEN_DUMP_DIR)
diff --git a/tools/Rules.mk b/tools/Rules.mk
index d9b9c740e9..abb1cbd783 100644
--- a/tools/Rules.mk
+++ b/tools/Rules.mk
@@ -194,7 +194,7 @@ subdirs-all subdirs-clean subdirs-install subdirs-distclean subdirs-uninstall: .
 		$(MAKE) subdir-$(patsubst subdirs-%,%,$@)-$$subdir; \
 	done
 
-subdir-all-% subdir-clean-% subdir-install-% subdir-uninstall-%: .phony
+subdir-all-% subdir-clean-% subdir-install-% subdir-uninstall-% subdir-run-%: .phony
 	$(MAKE) -C $* $(patsubst subdir-%-$*,%,$@)
 
 subdir-distclean-%: .phony
diff --git a/tools/tests/Makefile b/tools/tests/Makefile
index 6477a4386d..d73a6d7cfb 100644
--- a/tools/tests/Makefile
+++ b/tools/tests/Makefile
@@ -17,5 +17,23 @@ ifneq ($(clang),y)
 SUBDIRS-$(CONFIG_X86) += x86_emulator
 endif
 
-.PHONY: all clean install distclean uninstall
+.PHONY: all clean install distclean uninstall check
 all clean distclean install uninstall: %: subdirs-%
+
+# Subset of SUBDIRS that do not depend on xenctrl
+# Thus these tests can be run in the CI directly,
+# or on the developer's machine.
+
+SUBDIRS_CHECK-y :=
+SUBDIRS_CHECK-y += domid
+SUBDIRS_CHECK-y += pdx
+SUBDIRS_CHECK-y += rangeset
+SUBDIRS_CHECK-y += vpci
+
+SUBDIRS_CHECK-$(CONFIG_X86) += cpu-policy
+ifneq ($(clang),y)
+SUBDIRS_CHECK-$(CONFIG_X86) += x86_emulator
+endif
+
+# Ensure tests can run in parallel, the for loop in ../Rules.mk would serialize them
+check: $(foreach subdir,$(SUBDIRS_CHECK-y),subdir-run-$(subdir))
-- 
2.47.3

Re: [PATCH v1 3/5] tools/test: introduce a 'make check' rule

[Jan Beulich]

On 30.03.2026 18:17, Edwin Török wrote:
> --- a/tools/tests/Makefile
> +++ b/tools/tests/Makefile
> @@ -17,5 +17,23 @@ ifneq ($(clang),y)
>  SUBDIRS-$(CONFIG_X86) += x86_emulator
>  endif
>  
> -.PHONY: all clean install distclean uninstall
> +.PHONY: all clean install distclean uninstall check
>  all clean distclean install uninstall: %: subdirs-%
> +
> +# Subset of SUBDIRS that do not depend on xenctrl
> +# Thus these tests can be run in the CI directly,
> +# or on the developer's machine.
> +
> +SUBDIRS_CHECK-y :=
> +SUBDIRS_CHECK-y += domid
> +SUBDIRS_CHECK-y += pdx
> +SUBDIRS_CHECK-y += rangeset
> +SUBDIRS_CHECK-y += vpci
> +
> +SUBDIRS_CHECK-$(CONFIG_X86) += cpu-policy
> +ifneq ($(clang),y)
> +SUBDIRS_CHECK-$(CONFIG_X86) += x86_emulator
> +endif
> +
> +# Ensure tests can run in parallel, the for loop in ../Rules.mk would serialize them
> +check: $(foreach subdir,$(SUBDIRS_CHECK-y),subdir-run-$(subdir))

As said in reply to the earlier patch, the emulator harness is special.
While its building is suppressed (not causing a failure) when the toolchain
is too old, its running isn't suppressed like this. Instead, if you invoke
one of its two run* targets, it is assumed that you would override e.g. CC
to be capable of building the harness. (This is particularly important for
the run to actually have full coverage, since - as also said in the other
reply - blob building may fail with, at times, not bleeding edge tools.)

Perhaps its running here could be keyed to it having been built
successfully before. And then, if (on a 64-bit host) the 32-bit harness
was also successfully built, it probably should be run as well.

Jan

[PATCH v1 4/5] automation/scripts/build: run unit tests in the CI

[Edwin Török]

Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Link: https://lore.kernel.org/xen-devel/aa7V_nOG0ctnjxH3@macbook.local/
---
 automation/scripts/build | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/automation/scripts/build b/automation/scripts/build
index 359ee18444..26ffdf3350 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -107,6 +107,8 @@ else
     ./configure "${cfgargs[@]}"
     make -j$(nproc) dist
 
+    make -j$(nproc) check
+
     # Preserve artefacts
     # Note: Some smoke tests depending on finding binaries/xen on a full build
     # even though dist/ contains everything, while some containers don't even
-- 
2.47.3

[PATCH v1 5/5] automation/scripts/build: group command output from parallel jobs

[Edwin Török]

Whenever 'make -j' is used, '-O' should be used too.
This ensures that output from parallel invocations of the compiler or
tests do not mix with each-other.
The downside is that make has to buffer the output and will only print
it when the command completes, which can add a noticable delay during
local development, but should have no impact on the CI.

Signed-off-by: Edwin Török <edwin.torok@citrix.com>
---
 automation/scripts/build | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/automation/scripts/build b/automation/scripts/build
index 26ffdf3350..e30ac24e46 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -19,7 +19,7 @@ if [[ "${RANDCONFIG}" == "y" ]]; then
     # Append job-specific fixed configuration
     echo "${EXTRA_FIXED_RANDCONFIG}" >> xen/allrandom.config.tmp
 
-    make -j$(nproc) -C xen KCONFIG_ALLCONFIG=allrandom.config.tmp randconfig
+    make -j$(nproc) -O -C xen KCONFIG_ALLCONFIG=allrandom.config.tmp randconfig
 
     # RANDCONFIG implies HYPERVISOR_ONLY
     HYPERVISOR_ONLY="y"
@@ -34,7 +34,7 @@ else
         echo "${EXTRA_XEN_CONFIG}" >> xen/.config
     fi
 
-    make -j$(nproc) -C xen olddefconfig
+    make -j$(nproc) -O -C xen olddefconfig
 fi
 
 # Save the config file before building because build failure causes the script
@@ -64,7 +64,7 @@ if [[ "${CPPCHECK}" == "y" ]] && [[ "${HYPERVISOR_ONLY}" == "y" ]]; then
     cp xen/cppcheck-report/xen-cppcheck.txt xen-cppcheck.txt
 elif [[ "${HYPERVISOR_ONLY}" == "y" ]]; then
     # Xen-only build
-    make -j$(nproc) xen
+    make -j$(nproc) -O xen
 
     # Preserve artefacts
     collect_xen_artefacts
@@ -105,9 +105,9 @@ else
     fi
 
     ./configure "${cfgargs[@]}"
-    make -j$(nproc) dist
+    make -j$(nproc) -O dist
 
-    make -j$(nproc) check
+    make -j$(nproc) -O check
 
     # Preserve artefacts
     # Note: Some smoke tests depending on finding binaries/xen on a full build
-- 
2.47.3