[PATCH xenomai-images 0/3] longterm EVL testing under load

[PATCH xenomai-images 0/3] longterm EVL testing under load

[Tobias Schaffner]

Hi all,

This series introduces EVL test runs with additional stress applied via
stress-ng, as well as long-term testing. Along the way, I also updated the
long-term kernel under test from 6.12 to 6.18.

Please note:
 * The clock-timer-periodic test currently fails as soon as stress is
   applied. I proposed "tests: skip clock-timer-periodic on virtual targets"
   to address this.
 * On the Raspberry Pi 4, even the 3-minute test exceeds the latency
   threshold with debugging disabled:
   latency threshold is exceeded (150000 >= 163.951), aborting.
   https://gitlab.com/Xenomai/xenomai-images/-/jobs/15353275174

Best,
Tobias

Tobias Schaffner (3):
  ci: use stress-ng to stress EVL tests
  ci: Update for longterm test runs from 6.12 to 6.18
  ci: add long-term EVL CI test pipeline

 .gitlab-ci.yml                  |  8 +++++
 ci/lib/gitlab-ci-base.yml       |  2 ++
 ci/lib/kernel/6_12.yml          | 45 ++++++++++++++++++++++++++--
 ci/lib/kernel/6_12_physical.yml | 52 ---------------------------------
 ci/lib/kernel/6_18.yml          | 42 ++------------------------
 ci/lib/kernel/6_18_physical.yml | 52 +++++++++++++++++++++++++++++++++
 ci/xenomai_3_next_long_term.yml |  4 +--
 ci/xenomai_4_evl.yml            |  2 +-
 ci/xenomai_4_next.yml           |  2 +-
 ci/xenomai_4_next_long_term.yml | 24 +++++++++++++++
 kas/opt/ci.yaml                 |  1 +
 11 files changed, 136 insertions(+), 98 deletions(-)
 delete mode 100644 ci/lib/kernel/6_12_physical.yml
 create mode 100644 ci/lib/kernel/6_18_physical.yml
 create mode 100644 ci/xenomai_4_next_long_term.yml

-- 
2.43.0

[PATCH xenomai-images 1/3] ci: use stress-ng to stress EVL tests

[Tobias Schaffner]

Use stress-ng to apply CPU, virtual memory, FPU register, and procfs load
during EVL tests, mimicking the stressors used in Xenomai 3.

Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
 ci/lib/gitlab-ci-base.yml | 2 ++
 ci/xenomai_4_evl.yml      | 2 +-
 ci/xenomai_4_next.yml     | 2 +-
 kas/opt/ci.yaml           | 1 +
 4 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/ci/lib/gitlab-ci-base.yml b/ci/lib/gitlab-ci-base.yml
index 0540628..a3359bb 100644
--- a/ci/lib/gitlab-ci-base.yml
+++ b/ci/lib/gitlab-ci-base.yml
@@ -25,6 +25,8 @@ variables:
   ADDITIONAL_TEST_TIMEOUT_MINUTES: 10
   TEST_MAXIMUM_LATENCY: 0
   TEST_COMMAND: "xeno-test -k -l 'dohell -m /tmp ${TEST_RUNTIME_SECONDS}' ${ADDITIONAL_XENO_TEST_ARGS}"
+  EVL_STRESS_COMMAND: stress-ng --cpu 1 --vm 1 --vm-bytes 64M --fp 1 --procfs 1
+  EVL_TEST_COMMAND: evl test --stress-cmd "${EVL_STRESS_COMMAND}" -k $(evl test -l | grep -Ev "${EVL_TESTS_SKIP_PATTERN}")
   SSTATE_LOCATION: "s3://xenomai-images-artifacts/sstate"
   SSTATE_MIRRORS: "file://.* ${SSTATE_LOCATION}/PATH;downloadfilename=PATH"
   SSTATE_MAX_AGE: 14d
diff --git a/ci/xenomai_4_evl.yml b/ci/xenomai_4_evl.yml
index 283a1fd..c6a92b8 100644
--- a/ci/xenomai_4_evl.yml
+++ b/ci/xenomai_4_evl.yml
@@ -29,4 +29,4 @@ variables:
   XENOMAI_VERSION: "4-next"
   BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
   CACHE_KERNEL: evl
-  TEST_COMMAND: "evl test -k $(evl test -l | grep -Ev \"${EVL_TESTS_SKIP_PATTERN}\")"
+  TEST_COMMAND: "${EVL_TEST_COMMAND}"
diff --git a/ci/xenomai_4_next.yml b/ci/xenomai_4_next.yml
index 6ef4b6a..7ecfbb5 100644
--- a/ci/xenomai_4_next.yml
+++ b/ci/xenomai_4_next.yml
@@ -21,4 +21,4 @@ variables:
   KERNEL_VERSION_WITH_QEMU: "head"
   BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
   CACHE_KERNEL: evl
-  TEST_COMMAND: "evl test -k $(evl test -l | grep -Ev \"${EVL_TESTS_SKIP_PATTERN}\")"
+  TEST_COMMAND: "${EVL_TEST_COMMAND}"
diff --git a/kas/opt/ci.yaml b/kas/opt/ci.yaml
index e1466a0..f9e4bb7 100644
--- a/kas/opt/ci.yaml
+++ b/kas/opt/ci.yaml
@@ -20,6 +20,7 @@ local_conf_header:
   xenomai-ci: |
     ROOTFS_POSTPROCESS_COMMAND:remove = "image_postprocess_sshd_key_regen"
     IMAGE_INSTALL += "customizations-ci"
+    IMAGE_PREINSTALL += "stress-ng"
     IMAGE_INSTALL:remove = "sshd-regen-keys"
     IMAGE_INSTALL:remove = "expand-on-first-boot"
     IMAGE_FSTYPES = "${@ 'ext4.gz' if d.getVar('MACHINE', True).startswith('qemu') else 'tar.gz' }"
-- 
2.43.0

Re: [PATCH xenomai-images 1/3] ci: use stress-ng to stress EVL tests

[Florian Bezdeka]

On Wed, 2026-07-15 at 11:28 +0200, Tobias Schaffner wrote:
> Use stress-ng to apply CPU, virtual memory, FPU register, and procfs load
> during EVL tests, mimicking the stressors used in Xenomai 3.
> 
> Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
> ---
>  ci/lib/gitlab-ci-base.yml | 2 ++
>  ci/xenomai_4_evl.yml      | 2 +-
>  ci/xenomai_4_next.yml     | 2 +-
>  kas/opt/ci.yaml           | 1 +
>  4 files changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/ci/lib/gitlab-ci-base.yml b/ci/lib/gitlab-ci-base.yml
> index 0540628..a3359bb 100644
> --- a/ci/lib/gitlab-ci-base.yml
> +++ b/ci/lib/gitlab-ci-base.yml
> @@ -25,6 +25,8 @@ variables:
>    ADDITIONAL_TEST_TIMEOUT_MINUTES: 10
>    TEST_MAXIMUM_LATENCY: 0
>    TEST_COMMAND: "xeno-test -k -l 'dohell -m /tmp ${TEST_RUNTIME_SECONDS}' ${ADDITIONAL_XENO_TEST_ARGS}"
> +  EVL_STRESS_COMMAND: stress-ng --cpu 1 --vm 1 --vm-bytes 64M --fp 1 --procfs 1
> +  EVL_TEST_COMMAND: evl test --stress-cmd "${EVL_STRESS_COMMAND}" -k $(evl test -l | grep -Ev "${EVL_TESTS_SKIP_PATTERN}")
>    SSTATE_LOCATION: "s3://xenomai-images-artifacts/sstate"
>    SSTATE_MIRRORS: "file://.* ${SSTATE_LOCATION}/PATH;downloadfilename=PATH"
>    SSTATE_MAX_AGE: 14d
> diff --git a/ci/xenomai_4_evl.yml b/ci/xenomai_4_evl.yml
> index 283a1fd..c6a92b8 100644
> --- a/ci/xenomai_4_evl.yml
> +++ b/ci/xenomai_4_evl.yml
> @@ -29,4 +29,4 @@ variables:
>    XENOMAI_VERSION: "4-next"
>    BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
>    CACHE_KERNEL: evl
> -  TEST_COMMAND: "evl test -k $(evl test -l | grep -Ev \"${EVL_TESTS_SKIP_PATTERN}\")"
> +  TEST_COMMAND: "${EVL_TEST_COMMAND}"
> diff --git a/ci/xenomai_4_next.yml b/ci/xenomai_4_next.yml
> index 6ef4b6a..7ecfbb5 100644
> --- a/ci/xenomai_4_next.yml
> +++ b/ci/xenomai_4_next.yml
> @@ -21,4 +21,4 @@ variables:
>    KERNEL_VERSION_WITH_QEMU: "head"
>    BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
>    CACHE_KERNEL: evl
> -  TEST_COMMAND: "evl test -k $(evl test -l | grep -Ev \"${EVL_TESTS_SKIP_PATTERN}\")"
> +  TEST_COMMAND: "${EVL_TEST_COMMAND}"
> diff --git a/kas/opt/ci.yaml b/kas/opt/ci.yaml
> index e1466a0..f9e4bb7 100644
> --- a/kas/opt/ci.yaml
> +++ b/kas/opt/ci.yaml
> @@ -20,6 +20,7 @@ local_conf_header:
>    xenomai-ci: |
>      ROOTFS_POSTPROCESS_COMMAND:remove = "image_postprocess_sshd_key_regen"
>      IMAGE_INSTALL += "customizations-ci"
> +    IMAGE_PREINSTALL += "stress-ng"

Hm... As the CI configuration is now requesting stress-ng it seems right
that it is pre-installed by the CI image.

OTOH I hate those loose dependencies. Maybe the (lib)evl test package
should pull it in instead? So that the dependency is handled at package
level. Someone else might like to install the test package and get all
dependencies accordingly.

To solve that properly it might make sense to make stress-ng the default
stressor. That would allow the packaging to properly depend on it,
instead of only "suggest" stress-ng.

>      IMAGE_INSTALL:remove = "sshd-regen-keys"
>      IMAGE_INSTALL:remove = "expand-on-first-boot"
>      IMAGE_FSTYPES = "${@ 'ext4.gz' if d.getVar('MACHINE', True).startswith('qemu') else 'tar.gz' }"
> -- 
> 2.43.0

Re: [PATCH xenomai-images 1/3] ci: use stress-ng to stress EVL tests

[Tobias Schaffner]

Hi Florian,

On 7/15/26 15:00, Florian Bezdeka wrote:
> On Wed, 2026-07-15 at 11:28 +0200, Tobias Schaffner wrote:
>> Use stress-ng to apply CPU, virtual memory, FPU register, and procfs load
>> during EVL tests, mimicking the stressors used in Xenomai 3.
>>
>> Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
>> ---
>>   ci/lib/gitlab-ci-base.yml | 2 ++
>>   ci/xenomai_4_evl.yml      | 2 +-
>>   ci/xenomai_4_next.yml     | 2 +-
>>   kas/opt/ci.yaml           | 1 +
>>   4 files changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/ci/lib/gitlab-ci-base.yml b/ci/lib/gitlab-ci-base.yml
>> index 0540628..a3359bb 100644
>> --- a/ci/lib/gitlab-ci-base.yml
>> +++ b/ci/lib/gitlab-ci-base.yml
>> @@ -25,6 +25,8 @@ variables:
>>     ADDITIONAL_TEST_TIMEOUT_MINUTES: 10
>>     TEST_MAXIMUM_LATENCY: 0
>>     TEST_COMMAND: "xeno-test -k -l 'dohell -m /tmp ${TEST_RUNTIME_SECONDS}' ${ADDITIONAL_XENO_TEST_ARGS}"
>> +  EVL_STRESS_COMMAND: stress-ng --cpu 1 --vm 1 --vm-bytes 64M --fp 1 --procfs 1
>> +  EVL_TEST_COMMAND: evl test --stress-cmd "${EVL_STRESS_COMMAND}" -k $(evl test -l | grep -Ev "${EVL_TESTS_SKIP_PATTERN}")
>>     SSTATE_LOCATION: "s3://xenomai-images-artifacts/sstate"
>>     SSTATE_MIRRORS: "file://.* ${SSTATE_LOCATION}/PATH;downloadfilename=PATH"
>>     SSTATE_MAX_AGE: 14d
>> diff --git a/ci/xenomai_4_evl.yml b/ci/xenomai_4_evl.yml
>> index 283a1fd..c6a92b8 100644
>> --- a/ci/xenomai_4_evl.yml
>> +++ b/ci/xenomai_4_evl.yml
>> @@ -29,4 +29,4 @@ variables:
>>     XENOMAI_VERSION: "4-next"
>>     BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
>>     CACHE_KERNEL: evl
>> -  TEST_COMMAND: "evl test -k $(evl test -l | grep -Ev \"${EVL_TESTS_SKIP_PATTERN}\")"
>> +  TEST_COMMAND: "${EVL_TEST_COMMAND}"
>> diff --git a/ci/xenomai_4_next.yml b/ci/xenomai_4_next.yml
>> index 6ef4b6a..7ecfbb5 100644
>> --- a/ci/xenomai_4_next.yml
>> +++ b/ci/xenomai_4_next.yml
>> @@ -21,4 +21,4 @@ variables:
>>     KERNEL_VERSION_WITH_QEMU: "head"
>>     BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
>>     CACHE_KERNEL: evl
>> -  TEST_COMMAND: "evl test -k $(evl test -l | grep -Ev \"${EVL_TESTS_SKIP_PATTERN}\")"
>> +  TEST_COMMAND: "${EVL_TEST_COMMAND}"
>> diff --git a/kas/opt/ci.yaml b/kas/opt/ci.yaml
>> index e1466a0..f9e4bb7 100644
>> --- a/kas/opt/ci.yaml
>> +++ b/kas/opt/ci.yaml
>> @@ -20,6 +20,7 @@ local_conf_header:
>>     xenomai-ci: |
>>       ROOTFS_POSTPROCESS_COMMAND:remove = "image_postprocess_sshd_key_regen"
>>       IMAGE_INSTALL += "customizations-ci"
>> +    IMAGE_PREINSTALL += "stress-ng"
> 
> Hm... As the CI configuration is now requesting stress-ng it seems right
> that it is pre-installed by the CI image.
> 
> OTOH I hate those loose dependencies. Maybe the (lib)evl test package
> should pull it in instead? So that the dependency is handled at package
> level. Someone else might like to install the test package and get all
> dependencies accordingly.
> 
> To solve that properly it might make sense to make stress-ng the default
> stressor. That would allow the packaging to properly depend on it,
> instead of only "suggest" stress-ng.

I see your point but the decision to not make it a dependency was
intended.

First stress-ng is a debugging and evaluation tool and we do not want to
have it installed in production most of the time.

Second there are indeed persons/projects that have specialized other
loads. OSADL for example is using unix-bench in its realtime farm.

But most important this is a huge dependency. The large number of
libs needed for all the implemented stressors adds up to a total of
~240MB. Imo that is to large for libevl which also targets embedded
systems.

Best,
Tobias

PS: Taking the chance to share that we now have two EVL boards in the
OSADL realtime farm. Running on my 7.0 version port that I had available
at the time but will get an update soon:

7.0.0 #1 SMP PREEMPT EVL Starfive/Visionfive 2
https://www.osadl.org/Long-term-latency-plot-of-system-in-rack.qa-3d-latencyplot-r9s8.0.html?shadow=0

7.0.0 #1 SMP PREEMPT EVL NN/Raspberry Pi 4 Model B
https://www.osadl.org/Long-term-latency-plot-of-system-in-rack.qa-3d-latencyplot-r9s8.0.html?shadow=1

> 
>>       IMAGE_INSTALL:remove = "sshd-regen-keys"
>>       IMAGE_INSTALL:remove = "expand-on-first-boot"
>>       IMAGE_FSTYPES = "${@ 'ext4.gz' if d.getVar('MACHINE', True).startswith('qemu') else 'tar.gz' }"
>> -- 
>> 2.43.0

[PATCH xenomai-images 2/3] ci: Update for longterm test runs from 6.12 to 6.18

[Tobias Schaffner]

Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
 ci/lib/kernel/6_12.yml          | 45 ++++++++++++++++++++++++++--
 ci/lib/kernel/6_12_physical.yml | 52 ---------------------------------
 ci/lib/kernel/6_18.yml          | 42 ++------------------------
 ci/lib/kernel/6_18_physical.yml | 52 +++++++++++++++++++++++++++++++++
 ci/xenomai_3_next_long_term.yml |  4 +--
 5 files changed, 99 insertions(+), 96 deletions(-)
 delete mode 100644 ci/lib/kernel/6_12_physical.yml
 create mode 100644 ci/lib/kernel/6_18_physical.yml

diff --git a/ci/lib/kernel/6_12.yml b/ci/lib/kernel/6_12.yml
index a9bed87..529d24d 100644
--- a/ci/lib/kernel/6_12.yml
+++ b/ci/lib/kernel/6_12.yml
@@ -9,9 +9,6 @@
 # SPDX-License-Identifier: MIT
 #
 
-include:
-  - local: '/ci/lib/kernel/6_12_physical.yml'
-
 build-6.12:qemu-amd64:
   extends: .build:qemu-amd64
   variables:
@@ -50,3 +47,45 @@ lava-test-6.12:qemu-arm64:
   extends: .lava-test:qemu-arm64
   variables:
     KERNEL_VERSION: "6.12"
+
+build-6.12:rpi4:
+  extends: .build:rpi4
+  variables:
+    LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
+    KERNEL_VERSION: "6.12"
+    CACHE_KERNEL: dovetail
+
+lava-test-6.12:rpi4:
+  needs: [ "build-6.12:rpi4" ]
+  extends: .lava-test:rpi4
+  variables:
+    KERNEL_VERSION: "6.12"
+    TEST_MAXIMUM_LATENCY: 150
+
+build-6.12:beagle-bone-black:
+  extends: .build:beagle-bone-black
+  variables:
+    LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
+    KERNEL_VERSION: "6.12"
+    CACHE_KERNEL: dovetail
+
+lava-test-6.12:beagle-bone-black:
+  needs: [ "build-6.12:beagle-bone-black" ]
+  extends: .lava-test:beagle-bone-black
+  variables:
+    KERNEL_VERSION: "6.12"
+    TEST_MAXIMUM_LATENCY: 150
+
+build-6.12:x86-64-efi:
+  extends: .build:x86-64-efi
+  variables:
+    LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
+    KERNEL_VERSION: "6.12"
+    CACHE_KERNEL: dovetail
+
+lava-test-6.12:x86-64-efi:
+  needs: [ "build-6.12:x86-64-efi" ]
+  extends: .lava-test:x86-64-efi
+  variables:
+    KERNEL_VERSION: "6.12"
+    TEST_MAXIMUM_LATENCY: 100
diff --git a/ci/lib/kernel/6_12_physical.yml b/ci/lib/kernel/6_12_physical.yml
deleted file mode 100644
index 86cd23c..0000000
--- a/ci/lib/kernel/6_12_physical.yml
+++ /dev/null
@@ -1,52 +0,0 @@
-#
-# Xenomai Real-Time System
-#
-# Copyright (c) Siemens AG, 2024-2025
-#
-# Authors:
-#  Jan Kiszka <jan.kiszka@siemens.com>
-#
-# SPDX-License-Identifier: MIT
-#
-
-build-6.12:rpi4:
-  extends: .build:rpi4
-  variables:
-    LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
-    KERNEL_VERSION: "6.12"
-    CACHE_KERNEL: dovetail
-
-lava-test-6.12:rpi4:
-  needs: [ "build-6.12:rpi4" ]
-  extends: .lava-test:rpi4
-  variables:
-    KERNEL_VERSION: "6.12"
-    TEST_MAXIMUM_LATENCY: 150
-
-build-6.12:beagle-bone-black:
-  extends: .build:beagle-bone-black
-  variables:
-    LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
-    KERNEL_VERSION: "6.12"
-    CACHE_KERNEL: dovetail
-
-lava-test-6.12:beagle-bone-black:
-  needs: [ "build-6.12:beagle-bone-black" ]
-  extends: .lava-test:beagle-bone-black
-  variables:
-    KERNEL_VERSION: "6.12"
-    TEST_MAXIMUM_LATENCY: 150
-
-build-6.12:x86-64-efi:
-  extends: .build:x86-64-efi
-  variables:
-    LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
-    KERNEL_VERSION: "6.12"
-    CACHE_KERNEL: dovetail
-
-lava-test-6.12:x86-64-efi:
-  needs: [ "build-6.12:x86-64-efi" ]
-  extends: .lava-test:x86-64-efi
-  variables:
-    KERNEL_VERSION: "6.12"
-    TEST_MAXIMUM_LATENCY: 100
diff --git a/ci/lib/kernel/6_18.yml b/ci/lib/kernel/6_18.yml
index 88e3915..5d7e409 100644
--- a/ci/lib/kernel/6_18.yml
+++ b/ci/lib/kernel/6_18.yml
@@ -9,6 +9,9 @@
 # SPDX-License-Identifier: MIT
 #
 
+include:
+  - local: '/ci/lib/kernel/6_18_physical.yml'
+
 build-6.18:qemu-amd64:
   extends: .build:qemu-amd64
   variables:
@@ -47,42 +50,3 @@ lava-test-6.18:qemu-arm64:
   extends: .lava-test:qemu-arm64
   variables:
     KERNEL_VERSION: "6.18"
-
-build-6.18:rpi4:
-  extends: .build:rpi4
-  variables:
-    LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
-    KERNEL_VERSION: "6.18"
-    CACHE_KERNEL: dovetail
-
-lava-test-6.18:rpi4:
-  needs: [ "build-6.18:rpi4" ]
-  extends: .lava-test:rpi4
-  variables:
-    KERNEL_VERSION: "6.18"
-
-build-6.18:beagle-bone-black:
-  extends: .build:beagle-bone-black
-  variables:
-    LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
-    KERNEL_VERSION: "6.18"
-    CACHE_KERNEL: dovetail
-
-lava-test-6.18:beagle-bone-black:
-  needs: [ "build-6.18:beagle-bone-black" ]
-  extends: .lava-test:beagle-bone-black
-  variables:
-    KERNEL_VERSION: "6.18"
-
-build-6.18:x86-64-efi:
-  extends: .build:x86-64-efi
-  variables:
-    LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
-    KERNEL_VERSION: "6.18"
-    CACHE_KERNEL: dovetail
-
-lava-test-6.18:x86-64-efi:
-  needs: [ "build-6.18:x86-64-efi" ]
-  extends: .lava-test:x86-64-efi
-  variables:
-    KERNEL_VERSION: "6.18"
diff --git a/ci/lib/kernel/6_18_physical.yml b/ci/lib/kernel/6_18_physical.yml
new file mode 100644
index 0000000..68b7892
--- /dev/null
+++ b/ci/lib/kernel/6_18_physical.yml
@@ -0,0 +1,52 @@
+#
+# Xenomai Real-Time System
+#
+# Copyright (c) Siemens AG, 2024-2026
+#
+# Authors:
+#  Jan Kiszka <jan.kiszka@siemens.com>
+#
+# SPDX-License-Identifier: MIT
+#
+
+build-6.18:rpi4:
+  extends: .build:rpi4
+  variables:
+    LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
+    KERNEL_VERSION: "6.18"
+    CACHE_KERNEL: dovetail
+
+lava-test-6.18:rpi4:
+  needs: [ "build-6.18:rpi4" ]
+  extends: .lava-test:rpi4
+  variables:
+    KERNEL_VERSION: "6.18"
+    TEST_MAXIMUM_LATENCY: 150
+
+build-6.18:beagle-bone-black:
+  extends: .build:beagle-bone-black
+  variables:
+    LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
+    KERNEL_VERSION: "6.18"
+    CACHE_KERNEL: dovetail
+
+lava-test-6.18:beagle-bone-black:
+  needs: [ "build-6.18:beagle-bone-black" ]
+  extends: .lava-test:beagle-bone-black
+  variables:
+    KERNEL_VERSION: "6.18"
+    TEST_MAXIMUM_LATENCY: 150
+
+build-6.18:x86-64-efi:
+  extends: .build:x86-64-efi
+  variables:
+    LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
+    KERNEL_VERSION: "6.18"
+    CACHE_KERNEL: dovetail
+
+lava-test-6.18:x86-64-efi:
+  needs: [ "build-6.18:x86-64-efi" ]
+  extends: .lava-test:x86-64-efi
+  variables:
+    KERNEL_VERSION: "6.18"
+    TEST_MAXIMUM_LATENCY: 100
diff --git a/ci/xenomai_3_next_long_term.yml b/ci/xenomai_3_next_long_term.yml
index da8749e..c22374c 100644
--- a/ci/xenomai_3_next_long_term.yml
+++ b/ci/xenomai_3_next_long_term.yml
@@ -11,11 +11,11 @@
 
 include:
   - local: '/ci/lib/gitlab-ci-base.yml'
-  - local: '/ci/lib/kernel/6_12_physical.yml'
+  - local: '/ci/lib/kernel/6_18_physical.yml'
 
 variables:
   XENOMAI_BUILD_OPTION: ":kas/opt/xenomai-next.yaml"
   XENOMAI_VERSION: "3-next"
-  KERNEL_VERSION_WITH_QEMU: "6.12"
+  KERNEL_VERSION_WITH_QEMU: "6.18"
   TEST_RUNTIME_SECONDS: 3600
   TEST_COMMAND: "xeno-test -k -l 'dohell -m /tmp ${TEST_RUNTIME_SECONDS}' ${ADDITIONAL_XENO_TEST_ARGS} -- -m ${TEST_MAXIMUM_LATENCY}"
-- 
2.43.0

[PATCH xenomai-images 3/3] ci: add long-term EVL CI test pipeline

[Tobias Schaffner]

Add a scheduled Xenomai 4-next long-term CI job with a dedicated
pipeline configuration running extended EVL stress and latency tests.

Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
 .gitlab-ci.yml                  |  8 ++++++++
 ci/xenomai_4_next_long_term.yml | 24 ++++++++++++++++++++++++
 2 files changed, 32 insertions(+)
 create mode 100644 ci/xenomai_4_next_long_term.yml

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index b3d025e..589db8e 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -91,6 +91,14 @@ xenomai-4/next:
     include:
       - local: '/ci/xenomai_4_next.yml'
 
+xenomai-4/next-long-term:
+  extends: .parent
+  trigger:
+    include:
+      - local: '/ci/xenomai_4_next_long_term.yml'
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "schedule" && $ONLY_RUN == $CI_JOB_NAME'
+
 xenomai-4/evl:
   extends: .parent
   trigger:
diff --git a/ci/xenomai_4_next_long_term.yml b/ci/xenomai_4_next_long_term.yml
new file mode 100644
index 0000000..906f23b
--- /dev/null
+++ b/ci/xenomai_4_next_long_term.yml
@@ -0,0 +1,24 @@
+#
+# Xenomai Real-Time System
+#
+# Copyright (c) Siemens AG, 2026
+#
+# Authors:
+#  Tobias Schaffner <tobias.schaffner@siemens.com>
+#
+# SPDX-License-Identifier: MIT
+#
+
+include:
+  - local: '/ci/lib/gitlab-ci-base.yml'
+  - local: '/ci/lib/kernel/6_18_physical.yml'
+
+variables:
+  XENOMAI_BUILD_OPTION: ":kas/opt/libevl-next.yaml"
+  XENOMAI_VERSION: "4-next"
+  BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
+  CACHE_KERNEL: evl
+  TEST_RUNTIME_SECONDS: 3600
+  TEST_COMMAND: >-
+    evl test --stress-cmd "${EVL_STRESS_COMMAND}" -k $(evl test -l | grep -Ev "${EVL_TESTS_SKIP_PATTERN}") &&
+    evl test --stress-cmd "${EVL_STRESS_COMMAND}" latmus -- -T ${TEST_RUNTIME_SECONDS} -A ${TEST_MAXIMUM_LATENCY}
-- 
2.43.0

Re: [PATCH xenomai-images 0/3] longterm EVL testing under load

[Jan Kiszka]

On 15.07.26 11:28, Tobias Schaffner wrote:
> Hi all,
> 
> This series introduces EVL test runs with additional stress applied via
> stress-ng, as well as long-term testing. Along the way, I also updated the
> long-term kernel under test from 6.12 to 6.18.
> 
> Please note:
>  * The clock-timer-periodic test currently fails as soon as stress is
>    applied. I proposed "tests: skip clock-timer-periodic on virtual targets"
>    to address this.
>  * On the Raspberry Pi 4, even the 3-minute test exceeds the latency
>    threshold with debugging disabled:
>    latency threshold is exceeded (150000 >= 163.951), aborting.
>    https://gitlab.com/Xenomai/xenomai-images/-/jobs/15353275174

That is 150 ms?! Sounds like a real bug - unless we enforce some thermal
throttling on the hardware. But we haven't seen that with xenomai 3 so far.

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

Re: [PATCH xenomai-images 0/3] longterm EVL testing under load

[Tobias Schaffner]

On 7/15/26 12:19, Jan Kiszka wrote:
> On 15.07.26 11:28, Tobias Schaffner wrote:
>> Hi all,
>>
>> This series introduces EVL test runs with additional stress applied via
>> stress-ng, as well as long-term testing. Along the way, I also updated the
>> long-term kernel under test from 6.12 to 6.18.
>>
>> Please note:
>>   * The clock-timer-periodic test currently fails as soon as stress is
>>     applied. I proposed "tests: skip clock-timer-periodic on virtual targets"
>>     to address this.
>>   * On the Raspberry Pi 4, even the 3-minute test exceeds the latency
>>     threshold with debugging disabled:
>>     latency threshold is exceeded (150000 >= 163.951), aborting.
>>     https://gitlab.com/Xenomai/xenomai-images/-/jobs/15353275174
> 
> That is 150 ms?! Sounds like a real bug - unless we enforce some thermal
> throttling on the hardware. But we haven't seen that with xenomai 3 so far.

Also looks bad to me.

We seem to be a bit worse than the Xenomai 3 results. But the results 
are not great on Xenomai 3 either.

Xenomai3 Kernel 6.18:
https://gitlab.com/Xenomai/xenomai-images/-/jobs/15351736635
WCL: 123.148

Xenomai3 Kernel 6.12 (last weekly running long term run):
https://gitlab.com/Xenomai/xenomai-images/-/jobs/15308563317
WCL 137.195

The oldest Pipeline I was able to find is 11 Month old:
https://gitlab.com/Xenomai/xenomai-images/-/jobs/11042799343
WCL: 124.430

Best,
Tobias

> Jan
> 

Re: [PATCH xenomai-images 0/3] longterm EVL testing under load

[Philippe Gerum]

Tobias Schaffner <tobias.schaffner@siemens.com> writes:

> Hi all,
>
> This series introduces EVL test runs with additional stress applied via
> stress-ng, as well as long-term testing. Along the way, I also updated the
> long-term kernel under test from 6.12 to 6.18.
>
> Please note:
>  * The clock-timer-periodic test currently fails as soon as stress is
>    applied. I proposed "tests: skip clock-timer-periodic on virtual targets"
>    to address this.
>  * On the Raspberry Pi 4, even the 3-minute test exceeds the latency
>    threshold with debugging disabled:
>    latency threshold is exceeded (150000 >= 163.951), aborting.
>    https://gitlab.com/Xenomai/xenomai-images/-/jobs/15353275174

We should not exceed 80 µs or so on pi4.

-- 
Philippe.

Re: [PATCH xenomai-images 0/3] longterm EVL testing under load

[Philippe Gerum]

Tobias Schaffner <tobias.schaffner@siemens.com> writes:

>    latency threshold is exceeded (150000 >= 163.951), aborting.

Even the printout looks wrong.. Fix pushed upstream.

-- 
Philippe.