* [PATCH xenomai-images 0/3] longterm EVL testing under load
@ 2026-07-15 9:28 Tobias Schaffner
2026-07-15 9:28 ` [PATCH xenomai-images 1/3] ci: use stress-ng to stress EVL tests Tobias Schaffner
` (5 more replies)
0 siblings, 6 replies; 9+ messages in thread
From: Tobias Schaffner @ 2026-07-15 9:28 UTC (permalink / raw)
To: xenomai; +Cc: rpm, jan.kiszka, Tobias Schaffner
Hi all,
This series introduces EVL test runs with additional stress applied via
stress-ng, as well as long-term testing. Along the way, I also updated the
long-term kernel under test from 6.12 to 6.18.
Please note:
* The clock-timer-periodic test currently fails as soon as stress is
applied. I proposed "tests: skip clock-timer-periodic on virtual targets"
to address this.
* On the Raspberry Pi 4, even the 3-minute test exceeds the latency
threshold with debugging disabled:
latency threshold is exceeded (150000 >= 163.951), aborting.
https://gitlab.com/Xenomai/xenomai-images/-/jobs/15353275174
Best,
Tobias
Tobias Schaffner (3):
ci: use stress-ng to stress EVL tests
ci: Update for longterm test runs from 6.12 to 6.18
ci: add long-term EVL CI test pipeline
.gitlab-ci.yml | 8 +++++
ci/lib/gitlab-ci-base.yml | 2 ++
ci/lib/kernel/6_12.yml | 45 ++++++++++++++++++++++++++--
ci/lib/kernel/6_12_physical.yml | 52 ---------------------------------
ci/lib/kernel/6_18.yml | 42 ++------------------------
ci/lib/kernel/6_18_physical.yml | 52 +++++++++++++++++++++++++++++++++
ci/xenomai_3_next_long_term.yml | 4 +--
ci/xenomai_4_evl.yml | 2 +-
ci/xenomai_4_next.yml | 2 +-
ci/xenomai_4_next_long_term.yml | 24 +++++++++++++++
kas/opt/ci.yaml | 1 +
11 files changed, 136 insertions(+), 98 deletions(-)
delete mode 100644 ci/lib/kernel/6_12_physical.yml
create mode 100644 ci/lib/kernel/6_18_physical.yml
create mode 100644 ci/xenomai_4_next_long_term.yml
--
2.43.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH xenomai-images 1/3] ci: use stress-ng to stress EVL tests
2026-07-15 9:28 [PATCH xenomai-images 0/3] longterm EVL testing under load Tobias Schaffner
@ 2026-07-15 9:28 ` Tobias Schaffner
2026-07-15 13:00 ` Florian Bezdeka
2026-07-15 9:28 ` [PATCH xenomai-images 2/3] ci: Update for longterm test runs from 6.12 to 6.18 Tobias Schaffner
` (4 subsequent siblings)
5 siblings, 1 reply; 9+ messages in thread
From: Tobias Schaffner @ 2026-07-15 9:28 UTC (permalink / raw)
To: xenomai; +Cc: rpm, jan.kiszka, Tobias Schaffner
Use stress-ng to apply CPU, virtual memory, FPU register, and procfs load
during EVL tests, mimicking the stressors used in Xenomai 3.
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
ci/lib/gitlab-ci-base.yml | 2 ++
ci/xenomai_4_evl.yml | 2 +-
ci/xenomai_4_next.yml | 2 +-
kas/opt/ci.yaml | 1 +
4 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/ci/lib/gitlab-ci-base.yml b/ci/lib/gitlab-ci-base.yml
index 0540628..a3359bb 100644
--- a/ci/lib/gitlab-ci-base.yml
+++ b/ci/lib/gitlab-ci-base.yml
@@ -25,6 +25,8 @@ variables:
ADDITIONAL_TEST_TIMEOUT_MINUTES: 10
TEST_MAXIMUM_LATENCY: 0
TEST_COMMAND: "xeno-test -k -l 'dohell -m /tmp ${TEST_RUNTIME_SECONDS}' ${ADDITIONAL_XENO_TEST_ARGS}"
+ EVL_STRESS_COMMAND: stress-ng --cpu 1 --vm 1 --vm-bytes 64M --fp 1 --procfs 1
+ EVL_TEST_COMMAND: evl test --stress-cmd "${EVL_STRESS_COMMAND}" -k $(evl test -l | grep -Ev "${EVL_TESTS_SKIP_PATTERN}")
SSTATE_LOCATION: "s3://xenomai-images-artifacts/sstate"
SSTATE_MIRRORS: "file://.* ${SSTATE_LOCATION}/PATH;downloadfilename=PATH"
SSTATE_MAX_AGE: 14d
diff --git a/ci/xenomai_4_evl.yml b/ci/xenomai_4_evl.yml
index 283a1fd..c6a92b8 100644
--- a/ci/xenomai_4_evl.yml
+++ b/ci/xenomai_4_evl.yml
@@ -29,4 +29,4 @@ variables:
XENOMAI_VERSION: "4-next"
BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
CACHE_KERNEL: evl
- TEST_COMMAND: "evl test -k $(evl test -l | grep -Ev \"${EVL_TESTS_SKIP_PATTERN}\")"
+ TEST_COMMAND: "${EVL_TEST_COMMAND}"
diff --git a/ci/xenomai_4_next.yml b/ci/xenomai_4_next.yml
index 6ef4b6a..7ecfbb5 100644
--- a/ci/xenomai_4_next.yml
+++ b/ci/xenomai_4_next.yml
@@ -21,4 +21,4 @@ variables:
KERNEL_VERSION_WITH_QEMU: "head"
BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
CACHE_KERNEL: evl
- TEST_COMMAND: "evl test -k $(evl test -l | grep -Ev \"${EVL_TESTS_SKIP_PATTERN}\")"
+ TEST_COMMAND: "${EVL_TEST_COMMAND}"
diff --git a/kas/opt/ci.yaml b/kas/opt/ci.yaml
index e1466a0..f9e4bb7 100644
--- a/kas/opt/ci.yaml
+++ b/kas/opt/ci.yaml
@@ -20,6 +20,7 @@ local_conf_header:
xenomai-ci: |
ROOTFS_POSTPROCESS_COMMAND:remove = "image_postprocess_sshd_key_regen"
IMAGE_INSTALL += "customizations-ci"
+ IMAGE_PREINSTALL += "stress-ng"
IMAGE_INSTALL:remove = "sshd-regen-keys"
IMAGE_INSTALL:remove = "expand-on-first-boot"
IMAGE_FSTYPES = "${@ 'ext4.gz' if d.getVar('MACHINE', True).startswith('qemu') else 'tar.gz' }"
--
2.43.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH xenomai-images 2/3] ci: Update for longterm test runs from 6.12 to 6.18
2026-07-15 9:28 [PATCH xenomai-images 0/3] longterm EVL testing under load Tobias Schaffner
2026-07-15 9:28 ` [PATCH xenomai-images 1/3] ci: use stress-ng to stress EVL tests Tobias Schaffner
@ 2026-07-15 9:28 ` Tobias Schaffner
2026-07-15 9:28 ` [PATCH xenomai-images 3/3] ci: add long-term EVL CI test pipeline Tobias Schaffner
` (3 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Tobias Schaffner @ 2026-07-15 9:28 UTC (permalink / raw)
To: xenomai; +Cc: rpm, jan.kiszka, Tobias Schaffner
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
ci/lib/kernel/6_12.yml | 45 ++++++++++++++++++++++++++--
ci/lib/kernel/6_12_physical.yml | 52 ---------------------------------
ci/lib/kernel/6_18.yml | 42 ++------------------------
ci/lib/kernel/6_18_physical.yml | 52 +++++++++++++++++++++++++++++++++
ci/xenomai_3_next_long_term.yml | 4 +--
5 files changed, 99 insertions(+), 96 deletions(-)
delete mode 100644 ci/lib/kernel/6_12_physical.yml
create mode 100644 ci/lib/kernel/6_18_physical.yml
diff --git a/ci/lib/kernel/6_12.yml b/ci/lib/kernel/6_12.yml
index a9bed87..529d24d 100644
--- a/ci/lib/kernel/6_12.yml
+++ b/ci/lib/kernel/6_12.yml
@@ -9,9 +9,6 @@
# SPDX-License-Identifier: MIT
#
-include:
- - local: '/ci/lib/kernel/6_12_physical.yml'
-
build-6.12:qemu-amd64:
extends: .build:qemu-amd64
variables:
@@ -50,3 +47,45 @@ lava-test-6.12:qemu-arm64:
extends: .lava-test:qemu-arm64
variables:
KERNEL_VERSION: "6.12"
+
+build-6.12:rpi4:
+ extends: .build:rpi4
+ variables:
+ LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
+ KERNEL_VERSION: "6.12"
+ CACHE_KERNEL: dovetail
+
+lava-test-6.12:rpi4:
+ needs: [ "build-6.12:rpi4" ]
+ extends: .lava-test:rpi4
+ variables:
+ KERNEL_VERSION: "6.12"
+ TEST_MAXIMUM_LATENCY: 150
+
+build-6.12:beagle-bone-black:
+ extends: .build:beagle-bone-black
+ variables:
+ LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
+ KERNEL_VERSION: "6.12"
+ CACHE_KERNEL: dovetail
+
+lava-test-6.12:beagle-bone-black:
+ needs: [ "build-6.12:beagle-bone-black" ]
+ extends: .lava-test:beagle-bone-black
+ variables:
+ KERNEL_VERSION: "6.12"
+ TEST_MAXIMUM_LATENCY: 150
+
+build-6.12:x86-64-efi:
+ extends: .build:x86-64-efi
+ variables:
+ LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
+ KERNEL_VERSION: "6.12"
+ CACHE_KERNEL: dovetail
+
+lava-test-6.12:x86-64-efi:
+ needs: [ "build-6.12:x86-64-efi" ]
+ extends: .lava-test:x86-64-efi
+ variables:
+ KERNEL_VERSION: "6.12"
+ TEST_MAXIMUM_LATENCY: 100
diff --git a/ci/lib/kernel/6_12_physical.yml b/ci/lib/kernel/6_12_physical.yml
deleted file mode 100644
index 86cd23c..0000000
--- a/ci/lib/kernel/6_12_physical.yml
+++ /dev/null
@@ -1,52 +0,0 @@
-#
-# Xenomai Real-Time System
-#
-# Copyright (c) Siemens AG, 2024-2025
-#
-# Authors:
-# Jan Kiszka <jan.kiszka@siemens.com>
-#
-# SPDX-License-Identifier: MIT
-#
-
-build-6.12:rpi4:
- extends: .build:rpi4
- variables:
- LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
- KERNEL_VERSION: "6.12"
- CACHE_KERNEL: dovetail
-
-lava-test-6.12:rpi4:
- needs: [ "build-6.12:rpi4" ]
- extends: .lava-test:rpi4
- variables:
- KERNEL_VERSION: "6.12"
- TEST_MAXIMUM_LATENCY: 150
-
-build-6.12:beagle-bone-black:
- extends: .build:beagle-bone-black
- variables:
- LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
- KERNEL_VERSION: "6.12"
- CACHE_KERNEL: dovetail
-
-lava-test-6.12:beagle-bone-black:
- needs: [ "build-6.12:beagle-bone-black" ]
- extends: .lava-test:beagle-bone-black
- variables:
- KERNEL_VERSION: "6.12"
- TEST_MAXIMUM_LATENCY: 150
-
-build-6.12:x86-64-efi:
- extends: .build:x86-64-efi
- variables:
- LINUX_BUILD_OPTION: ":kas/opt/linux-6.12-head.yaml"
- KERNEL_VERSION: "6.12"
- CACHE_KERNEL: dovetail
-
-lava-test-6.12:x86-64-efi:
- needs: [ "build-6.12:x86-64-efi" ]
- extends: .lava-test:x86-64-efi
- variables:
- KERNEL_VERSION: "6.12"
- TEST_MAXIMUM_LATENCY: 100
diff --git a/ci/lib/kernel/6_18.yml b/ci/lib/kernel/6_18.yml
index 88e3915..5d7e409 100644
--- a/ci/lib/kernel/6_18.yml
+++ b/ci/lib/kernel/6_18.yml
@@ -9,6 +9,9 @@
# SPDX-License-Identifier: MIT
#
+include:
+ - local: '/ci/lib/kernel/6_18_physical.yml'
+
build-6.18:qemu-amd64:
extends: .build:qemu-amd64
variables:
@@ -47,42 +50,3 @@ lava-test-6.18:qemu-arm64:
extends: .lava-test:qemu-arm64
variables:
KERNEL_VERSION: "6.18"
-
-build-6.18:rpi4:
- extends: .build:rpi4
- variables:
- LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
- KERNEL_VERSION: "6.18"
- CACHE_KERNEL: dovetail
-
-lava-test-6.18:rpi4:
- needs: [ "build-6.18:rpi4" ]
- extends: .lava-test:rpi4
- variables:
- KERNEL_VERSION: "6.18"
-
-build-6.18:beagle-bone-black:
- extends: .build:beagle-bone-black
- variables:
- LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
- KERNEL_VERSION: "6.18"
- CACHE_KERNEL: dovetail
-
-lava-test-6.18:beagle-bone-black:
- needs: [ "build-6.18:beagle-bone-black" ]
- extends: .lava-test:beagle-bone-black
- variables:
- KERNEL_VERSION: "6.18"
-
-build-6.18:x86-64-efi:
- extends: .build:x86-64-efi
- variables:
- LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
- KERNEL_VERSION: "6.18"
- CACHE_KERNEL: dovetail
-
-lava-test-6.18:x86-64-efi:
- needs: [ "build-6.18:x86-64-efi" ]
- extends: .lava-test:x86-64-efi
- variables:
- KERNEL_VERSION: "6.18"
diff --git a/ci/lib/kernel/6_18_physical.yml b/ci/lib/kernel/6_18_physical.yml
new file mode 100644
index 0000000..68b7892
--- /dev/null
+++ b/ci/lib/kernel/6_18_physical.yml
@@ -0,0 +1,52 @@
+#
+# Xenomai Real-Time System
+#
+# Copyright (c) Siemens AG, 2024-2026
+#
+# Authors:
+# Jan Kiszka <jan.kiszka@siemens.com>
+#
+# SPDX-License-Identifier: MIT
+#
+
+build-6.18:rpi4:
+ extends: .build:rpi4
+ variables:
+ LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
+ KERNEL_VERSION: "6.18"
+ CACHE_KERNEL: dovetail
+
+lava-test-6.18:rpi4:
+ needs: [ "build-6.18:rpi4" ]
+ extends: .lava-test:rpi4
+ variables:
+ KERNEL_VERSION: "6.18"
+ TEST_MAXIMUM_LATENCY: 150
+
+build-6.18:beagle-bone-black:
+ extends: .build:beagle-bone-black
+ variables:
+ LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
+ KERNEL_VERSION: "6.18"
+ CACHE_KERNEL: dovetail
+
+lava-test-6.18:beagle-bone-black:
+ needs: [ "build-6.18:beagle-bone-black" ]
+ extends: .lava-test:beagle-bone-black
+ variables:
+ KERNEL_VERSION: "6.18"
+ TEST_MAXIMUM_LATENCY: 150
+
+build-6.18:x86-64-efi:
+ extends: .build:x86-64-efi
+ variables:
+ LINUX_BUILD_OPTION: ":kas/opt/linux-6.18-head.yaml"
+ KERNEL_VERSION: "6.18"
+ CACHE_KERNEL: dovetail
+
+lava-test-6.18:x86-64-efi:
+ needs: [ "build-6.18:x86-64-efi" ]
+ extends: .lava-test:x86-64-efi
+ variables:
+ KERNEL_VERSION: "6.18"
+ TEST_MAXIMUM_LATENCY: 100
diff --git a/ci/xenomai_3_next_long_term.yml b/ci/xenomai_3_next_long_term.yml
index da8749e..c22374c 100644
--- a/ci/xenomai_3_next_long_term.yml
+++ b/ci/xenomai_3_next_long_term.yml
@@ -11,11 +11,11 @@
include:
- local: '/ci/lib/gitlab-ci-base.yml'
- - local: '/ci/lib/kernel/6_12_physical.yml'
+ - local: '/ci/lib/kernel/6_18_physical.yml'
variables:
XENOMAI_BUILD_OPTION: ":kas/opt/xenomai-next.yaml"
XENOMAI_VERSION: "3-next"
- KERNEL_VERSION_WITH_QEMU: "6.12"
+ KERNEL_VERSION_WITH_QEMU: "6.18"
TEST_RUNTIME_SECONDS: 3600
TEST_COMMAND: "xeno-test -k -l 'dohell -m /tmp ${TEST_RUNTIME_SECONDS}' ${ADDITIONAL_XENO_TEST_ARGS} -- -m ${TEST_MAXIMUM_LATENCY}"
--
2.43.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH xenomai-images 3/3] ci: add long-term EVL CI test pipeline
2026-07-15 9:28 [PATCH xenomai-images 0/3] longterm EVL testing under load Tobias Schaffner
2026-07-15 9:28 ` [PATCH xenomai-images 1/3] ci: use stress-ng to stress EVL tests Tobias Schaffner
2026-07-15 9:28 ` [PATCH xenomai-images 2/3] ci: Update for longterm test runs from 6.12 to 6.18 Tobias Schaffner
@ 2026-07-15 9:28 ` Tobias Schaffner
2026-07-15 10:19 ` [PATCH xenomai-images 0/3] longterm EVL testing under load Jan Kiszka
` (2 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Tobias Schaffner @ 2026-07-15 9:28 UTC (permalink / raw)
To: xenomai; +Cc: rpm, jan.kiszka, Tobias Schaffner
Add a scheduled Xenomai 4-next long-term CI job with a dedicated
pipeline configuration running extended EVL stress and latency tests.
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
.gitlab-ci.yml | 8 ++++++++
ci/xenomai_4_next_long_term.yml | 24 ++++++++++++++++++++++++
2 files changed, 32 insertions(+)
create mode 100644 ci/xenomai_4_next_long_term.yml
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index b3d025e..589db8e 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -91,6 +91,14 @@ xenomai-4/next:
include:
- local: '/ci/xenomai_4_next.yml'
+xenomai-4/next-long-term:
+ extends: .parent
+ trigger:
+ include:
+ - local: '/ci/xenomai_4_next_long_term.yml'
+ rules:
+ - if: '$CI_PIPELINE_SOURCE == "schedule" && $ONLY_RUN == $CI_JOB_NAME'
+
xenomai-4/evl:
extends: .parent
trigger:
diff --git a/ci/xenomai_4_next_long_term.yml b/ci/xenomai_4_next_long_term.yml
new file mode 100644
index 0000000..906f23b
--- /dev/null
+++ b/ci/xenomai_4_next_long_term.yml
@@ -0,0 +1,24 @@
+#
+# Xenomai Real-Time System
+#
+# Copyright (c) Siemens AG, 2026
+#
+# Authors:
+# Tobias Schaffner <tobias.schaffner@siemens.com>
+#
+# SPDX-License-Identifier: MIT
+#
+
+include:
+ - local: '/ci/lib/gitlab-ci-base.yml'
+ - local: '/ci/lib/kernel/6_18_physical.yml'
+
+variables:
+ XENOMAI_BUILD_OPTION: ":kas/opt/libevl-next.yaml"
+ XENOMAI_VERSION: "4-next"
+ BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
+ CACHE_KERNEL: evl
+ TEST_RUNTIME_SECONDS: 3600
+ TEST_COMMAND: >-
+ evl test --stress-cmd "${EVL_STRESS_COMMAND}" -k $(evl test -l | grep -Ev "${EVL_TESTS_SKIP_PATTERN}") &&
+ evl test --stress-cmd "${EVL_STRESS_COMMAND}" latmus -- -T ${TEST_RUNTIME_SECONDS} -A ${TEST_MAXIMUM_LATENCY}
--
2.43.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH xenomai-images 0/3] longterm EVL testing under load
2026-07-15 9:28 [PATCH xenomai-images 0/3] longterm EVL testing under load Tobias Schaffner
` (2 preceding siblings ...)
2026-07-15 9:28 ` [PATCH xenomai-images 3/3] ci: add long-term EVL CI test pipeline Tobias Schaffner
@ 2026-07-15 10:19 ` Jan Kiszka
2026-07-15 10:46 ` Tobias Schaffner
2026-07-15 10:23 ` Philippe Gerum
2026-07-15 12:38 ` Philippe Gerum
5 siblings, 1 reply; 9+ messages in thread
From: Jan Kiszka @ 2026-07-15 10:19 UTC (permalink / raw)
To: Tobias Schaffner, xenomai; +Cc: rpm
On 15.07.26 11:28, Tobias Schaffner wrote:
> Hi all,
>
> This series introduces EVL test runs with additional stress applied via
> stress-ng, as well as long-term testing. Along the way, I also updated the
> long-term kernel under test from 6.12 to 6.18.
>
> Please note:
> * The clock-timer-periodic test currently fails as soon as stress is
> applied. I proposed "tests: skip clock-timer-periodic on virtual targets"
> to address this.
> * On the Raspberry Pi 4, even the 3-minute test exceeds the latency
> threshold with debugging disabled:
> latency threshold is exceeded (150000 >= 163.951), aborting.
> https://gitlab.com/Xenomai/xenomai-images/-/jobs/15353275174
That is 150 ms?! Sounds like a real bug - unless we enforce some thermal
throttling on the hardware. But we haven't seen that with xenomai 3 so far.
Jan
--
Siemens AG, Foundational Technologies
Linux Expert Center
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH xenomai-images 0/3] longterm EVL testing under load
2026-07-15 9:28 [PATCH xenomai-images 0/3] longterm EVL testing under load Tobias Schaffner
` (3 preceding siblings ...)
2026-07-15 10:19 ` [PATCH xenomai-images 0/3] longterm EVL testing under load Jan Kiszka
@ 2026-07-15 10:23 ` Philippe Gerum
2026-07-15 12:38 ` Philippe Gerum
5 siblings, 0 replies; 9+ messages in thread
From: Philippe Gerum @ 2026-07-15 10:23 UTC (permalink / raw)
To: Tobias Schaffner; +Cc: xenomai, jan.kiszka
Tobias Schaffner <tobias.schaffner@siemens.com> writes:
> Hi all,
>
> This series introduces EVL test runs with additional stress applied via
> stress-ng, as well as long-term testing. Along the way, I also updated the
> long-term kernel under test from 6.12 to 6.18.
>
> Please note:
> * The clock-timer-periodic test currently fails as soon as stress is
> applied. I proposed "tests: skip clock-timer-periodic on virtual targets"
> to address this.
> * On the Raspberry Pi 4, even the 3-minute test exceeds the latency
> threshold with debugging disabled:
> latency threshold is exceeded (150000 >= 163.951), aborting.
> https://gitlab.com/Xenomai/xenomai-images/-/jobs/15353275174
We should not exceed 80 µs or so on pi4.
--
Philippe.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH xenomai-images 0/3] longterm EVL testing under load
2026-07-15 10:19 ` [PATCH xenomai-images 0/3] longterm EVL testing under load Jan Kiszka
@ 2026-07-15 10:46 ` Tobias Schaffner
0 siblings, 0 replies; 9+ messages in thread
From: Tobias Schaffner @ 2026-07-15 10:46 UTC (permalink / raw)
To: Jan Kiszka, xenomai; +Cc: rpm
On 7/15/26 12:19, Jan Kiszka wrote:
> On 15.07.26 11:28, Tobias Schaffner wrote:
>> Hi all,
>>
>> This series introduces EVL test runs with additional stress applied via
>> stress-ng, as well as long-term testing. Along the way, I also updated the
>> long-term kernel under test from 6.12 to 6.18.
>>
>> Please note:
>> * The clock-timer-periodic test currently fails as soon as stress is
>> applied. I proposed "tests: skip clock-timer-periodic on virtual targets"
>> to address this.
>> * On the Raspberry Pi 4, even the 3-minute test exceeds the latency
>> threshold with debugging disabled:
>> latency threshold is exceeded (150000 >= 163.951), aborting.
>> https://gitlab.com/Xenomai/xenomai-images/-/jobs/15353275174
>
> That is 150 ms?! Sounds like a real bug - unless we enforce some thermal
> throttling on the hardware. But we haven't seen that with xenomai 3 so far.
Also looks bad to me.
We seem to be a bit worse than the Xenomai 3 results. But the results
are not great on Xenomai 3 either.
Xenomai3 Kernel 6.18:
https://gitlab.com/Xenomai/xenomai-images/-/jobs/15351736635
WCL: 123.148
Xenomai3 Kernel 6.12 (last weekly running long term run):
https://gitlab.com/Xenomai/xenomai-images/-/jobs/15308563317
WCL 137.195
The oldest Pipeline I was able to find is 11 Month old:
https://gitlab.com/Xenomai/xenomai-images/-/jobs/11042799343
WCL: 124.430
Best,
Tobias
> Jan
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH xenomai-images 0/3] longterm EVL testing under load
2026-07-15 9:28 [PATCH xenomai-images 0/3] longterm EVL testing under load Tobias Schaffner
` (4 preceding siblings ...)
2026-07-15 10:23 ` Philippe Gerum
@ 2026-07-15 12:38 ` Philippe Gerum
5 siblings, 0 replies; 9+ messages in thread
From: Philippe Gerum @ 2026-07-15 12:38 UTC (permalink / raw)
To: Tobias Schaffner; +Cc: xenomai, jan.kiszka
Tobias Schaffner <tobias.schaffner@siemens.com> writes:
> latency threshold is exceeded (150000 >= 163.951), aborting.
Even the printout looks wrong.. Fix pushed upstream.
--
Philippe.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH xenomai-images 1/3] ci: use stress-ng to stress EVL tests
2026-07-15 9:28 ` [PATCH xenomai-images 1/3] ci: use stress-ng to stress EVL tests Tobias Schaffner
@ 2026-07-15 13:00 ` Florian Bezdeka
0 siblings, 0 replies; 9+ messages in thread
From: Florian Bezdeka @ 2026-07-15 13:00 UTC (permalink / raw)
To: Tobias Schaffner, xenomai; +Cc: rpm, jan.kiszka
On Wed, 2026-07-15 at 11:28 +0200, Tobias Schaffner wrote:
> Use stress-ng to apply CPU, virtual memory, FPU register, and procfs load
> during EVL tests, mimicking the stressors used in Xenomai 3.
>
> Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
> ---
> ci/lib/gitlab-ci-base.yml | 2 ++
> ci/xenomai_4_evl.yml | 2 +-
> ci/xenomai_4_next.yml | 2 +-
> kas/opt/ci.yaml | 1 +
> 4 files changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/ci/lib/gitlab-ci-base.yml b/ci/lib/gitlab-ci-base.yml
> index 0540628..a3359bb 100644
> --- a/ci/lib/gitlab-ci-base.yml
> +++ b/ci/lib/gitlab-ci-base.yml
> @@ -25,6 +25,8 @@ variables:
> ADDITIONAL_TEST_TIMEOUT_MINUTES: 10
> TEST_MAXIMUM_LATENCY: 0
> TEST_COMMAND: "xeno-test -k -l 'dohell -m /tmp ${TEST_RUNTIME_SECONDS}' ${ADDITIONAL_XENO_TEST_ARGS}"
> + EVL_STRESS_COMMAND: stress-ng --cpu 1 --vm 1 --vm-bytes 64M --fp 1 --procfs 1
> + EVL_TEST_COMMAND: evl test --stress-cmd "${EVL_STRESS_COMMAND}" -k $(evl test -l | grep -Ev "${EVL_TESTS_SKIP_PATTERN}")
> SSTATE_LOCATION: "s3://xenomai-images-artifacts/sstate"
> SSTATE_MIRRORS: "file://.* ${SSTATE_LOCATION}/PATH;downloadfilename=PATH"
> SSTATE_MAX_AGE: 14d
> diff --git a/ci/xenomai_4_evl.yml b/ci/xenomai_4_evl.yml
> index 283a1fd..c6a92b8 100644
> --- a/ci/xenomai_4_evl.yml
> +++ b/ci/xenomai_4_evl.yml
> @@ -29,4 +29,4 @@ variables:
> XENOMAI_VERSION: "4-next"
> BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
> CACHE_KERNEL: evl
> - TEST_COMMAND: "evl test -k $(evl test -l | grep -Ev \"${EVL_TESTS_SKIP_PATTERN}\")"
> + TEST_COMMAND: "${EVL_TEST_COMMAND}"
> diff --git a/ci/xenomai_4_next.yml b/ci/xenomai_4_next.yml
> index 6ef4b6a..7ecfbb5 100644
> --- a/ci/xenomai_4_next.yml
> +++ b/ci/xenomai_4_next.yml
> @@ -21,4 +21,4 @@ variables:
> KERNEL_VERSION_WITH_QEMU: "head"
> BUILD_IDENTIFIER: "evl-${XENOMAI_VERSION}_kernel-${KERNEL_VERSION}"
> CACHE_KERNEL: evl
> - TEST_COMMAND: "evl test -k $(evl test -l | grep -Ev \"${EVL_TESTS_SKIP_PATTERN}\")"
> + TEST_COMMAND: "${EVL_TEST_COMMAND}"
> diff --git a/kas/opt/ci.yaml b/kas/opt/ci.yaml
> index e1466a0..f9e4bb7 100644
> --- a/kas/opt/ci.yaml
> +++ b/kas/opt/ci.yaml
> @@ -20,6 +20,7 @@ local_conf_header:
> xenomai-ci: |
> ROOTFS_POSTPROCESS_COMMAND:remove = "image_postprocess_sshd_key_regen"
> IMAGE_INSTALL += "customizations-ci"
> + IMAGE_PREINSTALL += "stress-ng"
Hm... As the CI configuration is now requesting stress-ng it seems right
that it is pre-installed by the CI image.
OTOH I hate those loose dependencies. Maybe the (lib)evl test package
should pull it in instead? So that the dependency is handled at package
level. Someone else might like to install the test package and get all
dependencies accordingly.
To solve that properly it might make sense to make stress-ng the default
stressor. That would allow the packaging to properly depend on it,
instead of only "suggest" stress-ng.
> IMAGE_INSTALL:remove = "sshd-regen-keys"
> IMAGE_INSTALL:remove = "expand-on-first-boot"
> IMAGE_FSTYPES = "${@ 'ext4.gz' if d.getVar('MACHINE', True).startswith('qemu') else 'tar.gz' }"
> --
> 2.43.0
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2026-07-15 13:01 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-15 9:28 [PATCH xenomai-images 0/3] longterm EVL testing under load Tobias Schaffner
2026-07-15 9:28 ` [PATCH xenomai-images 1/3] ci: use stress-ng to stress EVL tests Tobias Schaffner
2026-07-15 13:00 ` Florian Bezdeka
2026-07-15 9:28 ` [PATCH xenomai-images 2/3] ci: Update for longterm test runs from 6.12 to 6.18 Tobias Schaffner
2026-07-15 9:28 ` [PATCH xenomai-images 3/3] ci: add long-term EVL CI test pipeline Tobias Schaffner
2026-07-15 10:19 ` [PATCH xenomai-images 0/3] longterm EVL testing under load Jan Kiszka
2026-07-15 10:46 ` Tobias Schaffner
2026-07-15 10:23 ` Philippe Gerum
2026-07-15 12:38 ` Philippe Gerum
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.