* [PATCH RFC 0/8] Refactor CI
@ 2025-09-23 21:13 Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 1/8] base_image: optimize VM image copying with reflinks and fix same-file handling Daniel Gomez
` (8 more replies)
0 siblings, 9 replies; 10+ messages in thread
From: Daniel Gomez @ 2025-09-23 21:13 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez, dagomez
Refactor the .github/ CI setup into modular and reusable workflows and
actions. This makes it possible to share modules from kdevops repo and
reuse them in other GitHub projects.
Highlights:
* Add support for parallel workflows
* Add support for kdevops selftest CI with xfs_reflink_4k and
blktests_nvme (more coming)
* Run workflow tests daily with the latest linux-next tag and weekly
(Mondays) with the latest Linus tag
* Update commit format in kdevops-results-archive for both CI modes
(samples below)
* Enable reproducible builds and ccache. With a fixed kernel version,
this should reduce kernel build time. However, without fragments it does
not fully work yet. The open question is whether we should enable it
only for kdevops-ci or not for workflow testing. Can we trust ccache for
testing and reporting results? I think it's just fine for kdevops-ci
path.
* Add time metrics to identify bottlenecks and track improvements
(e.g., XFS reflink saved ~1 minute on
"base_image : Copy custom image to base image location"). Report example:
TASKS RECAP ********************************************************************
Tuesday 23 September 2025 19:23:01 +0000 (0:00:42.426) 0:00:42.426 *****
===============================================================================
base_image : Generating the index for debian-13-generic-amd64-daily ---- 28.65s
Gathering Facts --------------------------------------------------------- 4.29s
guestfs : Check for dnsmasq configuration files ------------------------- 0.79s
guestfs : Create storage pool path directory and set group (libvirt system uri) --- 0.60s
guestfs : Check if directory has group write permissions (libvirt system uri) --- 0.57s
guestfs : Get the user who invoked Ansible ------------------------------ 0.54s
guestfs : Check if directory is owned by the correct group (libvirt system uri) --- 0.51s
guestfs : Create kdevops guestfs storage directory (libvirt system uri) --- 0.50s
guestfs : Check if libvirt default network is running ------------------- 0.45s
base_image : Ensure the custom image directory exists ------------------- 0.43s
base_image : Stat /var/lib/libvirt/images/kdevops/guestfs/custom_images/debian-13-generic-amd64-daily/debian-13-generic-amd64-daily.raw --- 0.41s
base_image : Check if the custom image sentinel file already exists ----- 0.40s
base_image : Check if the custom image file already exists -------------- 0.40s
base_image : Check if the custom index exists --------------------------- 0.39s
guestfs : Check if dnsmasq service is enabled --------------------------- 0.38s
guestfs : Check if dnsmasq service is active ---------------------------- 0.36s
base_image : Check if the custom source exists -------------------------- 0.27s
base_image : Create custom upstream OS image ---------------------------- 0.11s
base_image : Copy custom image to base image location (with automatic reflink optimization) --- 0.09s
base_image : Set the pathname of the custom image ----------------------- 0.06s
Commit format for kdevops-ci:
kdevops-ci: xfs_reflink_4k: d03219f7808b ansible: add lightning symbol for changed task status
BUILD INFO:
kdevops: d03219f7808b ("ansible: add lightning symbol for changed task status")
Workflow: xfs_reflink_4k
Test: generic/003
Kernel: v6.15
Duration: 1m 44s
EXECUTION RESULTS:
KERNEL: 6.15.0
CPUS: 8
xfs_reflink_4k: 1 tests, 17 seconds
generic/003 Pass 14s
Totals: 1 tests, 0 skipped, 0 failures, 0 errors, 14s
METADATA:
workflow: xfs_reflink_4k | scope: kdevops | test: generic/003 | requested: v6.15 | actual: v6.15 | result: not ok
CPUS: 8
xfs_reflink_4k: 1 tests, 17 seconds
generic/003 Pass 14s
Totals: 1 tests, 0 skipped, 0 failures, 0 errors, 14s
Commit format for workflow validation on linux:
blktests (linux master): PASS
BUILD INFO:
Kernel: master ("Merge tag 'sched_ext-for-6.17-rc7-fixes' of
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext")
Workflow: blktests
kdevops: 2115e4296e10 ("base_image: optimize VM image copying with reflinks and fix
same-file handling")
Scope: full test suite
Duration: 35m 57s
EXECUTION RESULTS:
Kernel tests results:
Blktests summary:
Tests run: 81, Passed: 81, Failed: 0
Sample passed test:
NQN:blktests-subsystem-1 disconnected 1 controller(s)
NQN:blktests-subsystem-1 disconnected 1 controller(s)
METADATA:
workflow: blktests | tree: linux
ref: master | scope: tests | result: ok
Blktests summary:
Tests run: 81, Passed: 81, Failed: 0
Sample passed test:
NQN:blktests-subsystem-1 disconnected 1 controller(s)
NQN:blktests-subsystem-1 disconnected 1 controller(s) | requested: master | actual: master
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
Daniel Gomez (8):
base_image: optimize VM image copying with reflinks and fix same-file handling
kconfig: add KDEVOPS_MAKE_VERBOSE option for persistent verbose output
defconfig: add ci fragment
ansible: enhance DIY callback output format for improved readability
ansible: add profile_tasks callback timing analysis support
scripts: fix CI_WORKFLOW basename mapping
fstests: fix ansible_host undefined error
github: refactor into reusable actions and workflows
.github/actions/archive/action.yml | 28 +++
.github/actions/cleanup/action.yml | 24 ++
.github/actions/setup/action.yml | 187 +++++++++++++++
.github/actions/test/action.yml | 213 +++++++++++++++++
.github/workflows/destroy.yml | 93 ++++++++
.github/workflows/docker-tests.yml | 2 +-
.github/workflows/fstests.yml | 98 --------
.github/workflows/main.yml | 97 ++++++++
.github/workflows/manual.yml | 122 ++++++++++
.github/workflows/push.yml | 29 +++
.github/workflows/schedule.yml | 54 +++++
Makefile | 7 +
defconfigs/configs/ci.config | 7 +
kconfigs/Kconfig.ansible_cfg | 14 ++
kconfigs/Kconfig.kdevops | 18 ++
.../roles/ansible_cfg/templates/ansible.cfg.j2 | 20 +-
playbooks/roles/base_image/tasks/custom-image.yml | 18 +-
playbooks/roles/fstests/tasks/main.yml | 10 +-
scripts/ci.Makefile | 13 +
scripts/generate_ci_commit_message.sh | 263 +++++++++++++++++++++
scripts/github_output.sh | 9 +
scripts/korg-releases.py | 94 ++++++++
22 files changed, 1305 insertions(+), 115 deletions(-)
---
base-commit: 4b42d3d3636b6b0c04ccc72b3cd766abe1526344
change-id: 20250923-ci-refactor-9fd37649f4f3
Best regards,
--
Daniel Gomez <da.gomez@samsung.com>
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH RFC 1/8] base_image: optimize VM image copying with reflinks and fix same-file handling
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
@ 2025-09-23 21:13 ` Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 2/8] kconfig: add KDEVOPS_MAKE_VERBOSE option for persistent verbose output Daniel Gomez
` (7 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Daniel Gomez @ 2025-09-23 21:13 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez
From: Daniel Gomez <da.gomez@samsung.com>
Replace ansible.builtin.copy with cp --reflink=auto for significant
performance improvement on XFS filesystems with reflink support:
- 20GB image copy time: ~54s → ~1s (when reflinks are supported)
- Automatic fallback to regular copy on non-reflink filesystems
- Use direct cp command for better control over reflink behavior
Add condition to prevent copying when source and destination are the
same file. This was not needed with ansible.builtin.copy (which handles
this case automatically) but is required with direct cp command which
fails with "files are the same" error.
The condition `custom_image != base_image_pathname` ensures copy only
occurs when paths differ, maintaining compatibility with configurations
where both variables point to the same location.
Generated-by: Claude AI
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
playbooks/roles/base_image/tasks/custom-image.yml | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/playbooks/roles/base_image/tasks/custom-image.yml b/playbooks/roles/base_image/tasks/custom-image.yml
index bcf35933..bac834e8 100644
--- a/playbooks/roles/base_image/tasks/custom-image.yml
+++ b/playbooks/roles/base_image/tasks/custom-image.yml
@@ -338,13 +338,21 @@
- "{{ custom_image_dir }}"
changed_when: true
-- name: Copy custom image to base image location
+- name: Copy custom image to base image location (with automatic reflink optimization)
become: true
become_method: ansible.builtin.sudo
- ansible.builtin.copy:
- src: "{{ custom_image }}"
- dest: "{{ base_image_pathname }}"
- remote_src: true
+ ansible.builtin.command:
+ cmd: "cp --reflink=auto '{{ custom_image }}' '{{ base_image_pathname }}'"
+ when:
+ - custom_image_stat.stat.exists or custom_image_download is changed
+ - custom_image != base_image_pathname
+
+- name: Set proper permissions on base image
+ become: true
+ become_method: ansible.builtin.sudo
+ ansible.builtin.file:
+ path: "{{ base_image_pathname }}"
mode: "u=rw,g=r,o=r"
when:
- custom_image_stat.stat.exists or custom_image_download is changed
+ - custom_image != base_image_pathname
--
2.50.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH RFC 2/8] kconfig: add KDEVOPS_MAKE_VERBOSE option for persistent verbose output
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 1/8] base_image: optimize VM image copying with reflinks and fix same-file handling Daniel Gomez
@ 2025-09-23 21:13 ` Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 3/8] defconfig: add ci fragment Daniel Gomez
` (6 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Daniel Gomez @ 2025-09-23 21:13 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez, dagomez
From: Daniel Gomez <da.gomez@samsung.com>
Add a new kconfig option in the "Kdevops configuration" menu to enable
verbose make output (V=1) by default. This promotes make verbosity from
a command-line-only option to a first-class configuration setting.
Features:
- KDEVOPS_MAKE_VERBOSE bool option in menuconfig
- Early parsing in main Makefile before V= logic
- Command line override support with proper priority:
1. CLI V=0/V=1 (highest priority)
2. CONFIG_KDEVOPS_MAKE_VERBOSE=y (persistent default)
3. Built-in quiet mode (fallback)
Usage examples:
- make target → Uses kconfig setting
- make V=0 target → CLI overrides to quiet
- make V=1 target → CLI overrides to verbose
This allows users to set a persistent verbosity preference while
maintaining full command-line flexibility, following industry
standard patterns used by Rust/Cargo, Kubernetes, and other tools.
Generated-by: Claude AI
Signed-off-by: dagomez <dagomez@example.com>
---
Makefile | 7 +++++++
kconfigs/Kconfig.kdevops | 18 ++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/Makefile b/Makefile
index 90e8091d..9ea479c9 100644
--- a/Makefile
+++ b/Makefile
@@ -64,6 +64,13 @@ define print_target
echo "==> [$1]"
endef
+# Parse kconfig verbosity setting early, allow CLI to override
+ifneq (,$(wildcard .config))
+ifeq ($(CONFIG_KDEVOPS_MAKE_VERBOSE),y)
+V ?= 1
+endif
+endif
+
ifeq ($(V),1)
export Q=@$(call print_target,$@) && set -x &&
export NQ=true
diff --git a/kconfigs/Kconfig.kdevops b/kconfigs/Kconfig.kdevops
index c2362adf..d22edd01 100644
--- a/kconfigs/Kconfig.kdevops
+++ b/kconfigs/Kconfig.kdevops
@@ -76,6 +76,24 @@ config KDEVOPS_CUSTOM_SSH_KEXALGORITHMS
Some distributions, such as older distributions, may require a custom
ssh configuration entry for the KexAlgorithms parameter.
+config KDEVOPS_MAKE_VERBOSE
+ bool "Enable verbose make output (V=1)"
+ default n
+ help
+ Enable verbose output for all make operations. This is equivalent
+ to running 'make V=1' and shows all executed commands for debugging.
+
+ When enabled, you'll see:
+ - All shell commands being executed
+ - Full ansible-playbook command lines
+ - Detailed build output
+
+ This is useful for debugging build issues and understanding what
+ kdevops is doing behind the scenes. You can still override this
+ setting on the command line with 'make V=0' or 'make V=1'.
+
+ If unsure, say N.
+
config KDEVOPS_BASELINE_AND_DEV
bool "Enable both a baseline and development system per target test"
default n
--
2.50.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH RFC 3/8] defconfig: add ci fragment
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 1/8] base_image: optimize VM image copying with reflinks and fix same-file handling Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 2/8] kconfig: add KDEVOPS_MAKE_VERBOSE option for persistent verbose output Daniel Gomez
@ 2025-09-23 21:13 ` Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 4/8] ansible: enhance DIY callback output format for improved readability Daniel Gomez
` (5 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Daniel Gomez @ 2025-09-23 21:13 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez
From: Daniel Gomez <da.gomez@samsung.com>
Introduce a CI-specific fragment to enable the configuration options
required for continuous integration.
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
defconfigs/configs/ci.config | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/defconfigs/configs/ci.config b/defconfigs/configs/ci.config
new file mode 100644
index 00000000..00af941c
--- /dev/null
+++ b/defconfigs/configs/ci.config
@@ -0,0 +1,7 @@
+CONFIG_ANSIBLE_CFG_CALLBACK_PLUGIN_PROFILE_TASKS=y
+CONFIG_ANSIBLE_CFG_FORKS=30
+CONFIG_BOOTLINUX_CCACHE_SYSTEM_WIDE=y
+CONFIG_BOOTLINUX_CCACHE=y
+CONFIG_BOOTLINUX_CLEAN_BEFORE_BUILD=y
+CONFIG_BOOTLINUX_REPRODUCIBLE_BUILDS=y
+CONFIG_KDEVOPS_MAKE_VERBOSE=y
--
2.50.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH RFC 4/8] ansible: enhance DIY callback output format for improved readability
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
` (2 preceding siblings ...)
2025-09-23 21:13 ` [PATCH RFC 3/8] defconfig: add ci fragment Daniel Gomez
@ 2025-09-23 21:13 ` Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 5/8] ansible: add profile_tasks callback timing analysis support Daniel Gomez
` (4 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Daniel Gomez @ 2025-09-23 21:13 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez
From: Daniel Gomez <da.gomez@samsung.com>
Improve the DIY callback output format to provide cleaner, more scannable
task execution display:
- Show target hosts in task start messages using ansible_play_hosts
- Use visual status indicators: ✓ for success, ⊘ for skipped tasks
- Remove redundant task name repetition in completion messages
- Add indentation to status messages for better visual hierarchy
- Remove unnecessary "start:" messages to reduce noise
- Keep (changed) and FAILED messages with full context
Output format:
TASK: Task name [target,hosts]
✓ [actual_host]
⊘ [skipped_host]
This creates clean, scannable output similar to systemd service status
with clear visual indicators while showing host targeting information
upfront and execution results with proper visual hierarchy.
Generated-by: Claude AI
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
playbooks/roles/ansible_cfg/templates/ansible.cfg.j2 | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2 b/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2
index f3cc1797..c8981e09 100644
--- a/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2
+++ b/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2
@@ -17,17 +17,17 @@ inventory = {{ ansible_cfg_inventory }}
[callback_diy]
playbook_on_play_start_msg = PLAY: {{ '{{' }} ansible_callback_diy.play.name | default('') | upper {{ '}}' }}
playbook_on_play_start_msg_color = bright blue
-playbook_on_task_start_msg = TASK: {{ '{{' }} ansible_callback_diy.task.name | default('') {{ '}}' }}
+playbook_on_task_start_msg = TASK: {{ '{{' }} ansible_callback_diy.task.name | default('') {{ '}}' }} [{{ '{{' }} ansible_play_hosts | join(',') {{ '}}' }}]
playbook_on_task_start_msg_color = yellow
-runner_on_start_msg = start: [{{ '{{' }} ansible_callback_diy.host.name | default('host') {{ '}}' }}]
+runner_on_start_msg =
runner_on_start_msg_color = yellow
-runner_on_ok_msg = ok: [{{ '{{' }} ansible_callback_diy.result.host | default('host') {{ '}}' }}]
+runner_on_ok_msg = ⠀⠀✓ [{{ '{{' }} ansible_callback_diy.result.host | default('host') {{ '}}' }}]
runner_on_ok_msg_color = green
-runner_on_changed_msg = changed: [{{ '{{' }} ansible_callback_diy.result.host | default('host') {{ '}}' }}]
+runner_on_changed_msg = ⠀⠀(changed) [{{ '{{' }} ansible_callback_diy.result.host | default('host') {{ '}}' }}]
runner_on_changed_msg_color = bright yellow
-runner_on_failed_msg = FAILED: [{{ '{{' }} ansible_callback_diy.result.host | default('host') {{ '}}' }}] => {{ '{{' }} msg | default('') {{ '}}' }}{{ '{%' }} if stderr is defined and stderr | length > 0 {{ '%}' }} STDERR: {{ '{{' }} stderr {{ '}}' }}{{ '{%' }} endif {{ '%}' }}{{ '{%' }} if stdout is defined and stdout | length > 0 {{ '%}' }} STDOUT: {{ '{{' }} stdout {{ '}}' }}{{ '{%' }} endif {{ '%}' }}
+runner_on_failed_msg = ⠀⠀FAILED: [{{ '{{' }} ansible_callback_diy.result.host | default('host') {{ '}}' }}] => {{ '{{' }} msg | default('') {{ '}}' }}{{ '{%' }} if stderr is defined and stderr | length > 0 {{ '%}' }} STDERR: {{ '{{' }} stderr {{ '}}' }}{{ '{%' }} endif {{ '%}' }}{{ '{%' }} if stdout is defined and stdout | length > 0 {{ '%}' }} STDOUT: {{ '{{' }} stdout {{ '}}' }}{{ '{%' }} endif {{ '%}' }}
runner_on_failed_msg_color = bright red
-runner_on_skipped_msg = skipped: [{{ '{{' }} ansible_callback_diy.result.host | default('host') {{ '}}' }}]
+runner_on_skipped_msg = ⠀⠀⊘ [{{ '{{' }} ansible_callback_diy.result.host | default('host') {{ '}}' }}]
runner_on_skipped_msg_color = cyan
runner_on_unreachable_msg = UNREACHABLE: [{{ '{{' }} ansible_callback_diy.result.host | default('host') {{ '}}' }}]
runner_on_unreachable_msg_color = bright magenta
--
2.50.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH RFC 5/8] ansible: add profile_tasks callback timing analysis support
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
` (3 preceding siblings ...)
2025-09-23 21:13 ` [PATCH RFC 4/8] ansible: enhance DIY callback output format for improved readability Daniel Gomez
@ 2025-09-23 21:13 ` Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 6/8] scripts: fix CI_WORKFLOW basename mapping Daniel Gomez
` (3 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Daniel Gomez @ 2025-09-23 21:13 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez
From: Daniel Gomez <da.gomez@samsung.com>
Add new Kconfig option ANSIBLE_CFG_CALLBACK_PLUGIN_PROFILE_TASKS
to enable the ansible.posix.profile_tasks callback plugin.
This provides timing analysis similar to systemd-analyze blame,
showing task execution times and generating a TASKS RECAP section
with the slowest tasks at the end of playbook execution.
The option is disabled by default to maintain clean output for
existing workflows, but can be enabled when performance analysis
is needed.
Generated-by: Claude AI
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
kconfigs/Kconfig.ansible_cfg | 14 ++++++++++++++
playbooks/roles/ansible_cfg/templates/ansible.cfg.j2 | 8 ++++++++
2 files changed, 22 insertions(+)
diff --git a/kconfigs/Kconfig.ansible_cfg b/kconfigs/Kconfig.ansible_cfg
index 430c0e4c..c04532e8 100644
--- a/kconfigs/Kconfig.ansible_cfg
+++ b/kconfigs/Kconfig.ansible_cfg
@@ -156,6 +156,20 @@ config ANSIBLE_CFG_CALLBACK_PLUGIN_SHOW_TASK_PATH_ON_FAILURE
the line number. This information is displayed automatically for every task when
running with -vv or greater verbosity.
https://docs.ansible.com/ansible/latest/collections/community/general/dense_callback.html#parameter-show_task_path_on_failure
+
+config ANSIBLE_CFG_CALLBACK_PLUGIN_PROFILE_TASKS
+ bool "Enable profile_tasks callback for timing analysis"
+ output yaml
+ default n
+ help
+ Enable the profile_tasks callback plugin to provide timing analysis
+ similar to systemd-analyze blame. This shows how long each task takes
+ to execute and provides a summary of the slowest tasks at the end.
+
+ When enabled, this adds timing information to playbook execution
+ and generates a "TASKS RECAP" section showing execution times.
+
+ https://docs.ansible.com/ansible/latest/collections/ansible/posix/profile_tasks_callback.html
endmenu
config ANSIBLE_CFG_DEPRECATION_WARNINGS
diff --git a/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2 b/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2
index c8981e09..f3f6c723 100644
--- a/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2
+++ b/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2
@@ -1,4 +1,7 @@
[defaults]
+{% if ansible_cfg_callback_plugin_profile_tasks %}
+callbacks_enabled = ansible.posix.profile_tasks, {{ ansible_cfg_callback_plugin_string }}
+{% endif %}
deprecation_warnings = {{ ansible_cfg_deprecation_warnings }}
stdout_callback = {{ ansible_cfg_callback_plugin_string }}
check_mode_markers = {{ ansible_cfg_callback_plugin_check_mode_markers }}
@@ -39,6 +42,11 @@ playbook_on_stats_msg = PLAYBOOK SUMMARY
{{ '{%' }} endfor {{ '%}' }}
playbook_on_stats_msg_color = bright green
{% endif %}
+{% if ansible_cfg_callback_plugin_profile_tasks %}
+
+[callback_profile_tasks]
+summary_only = true
+{% endif %}
{% if ansible_facts['distribution'] == 'openSUSE' %}
[connection]
retries = {{ ansible_cfg_reconnection_retries }}
--
2.50.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH RFC 6/8] scripts: fix CI_WORKFLOW basename mapping
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
` (4 preceding siblings ...)
2025-09-23 21:13 ` [PATCH RFC 5/8] ansible: add profile_tasks callback timing analysis support Daniel Gomez
@ 2025-09-23 21:13 ` Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 7/8] fstests: fix ansible_host undefined error Daniel Gomez
` (2 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Daniel Gomez @ 2025-09-23 21:13 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez
From: Daniel Gomez <da.gomez@samsung.com>
Fix CI_WORKFLOW basename mapping in scripts/ci.Makefile:
- Add mapping for XFS/fstests workflows
- Map xfs*/btrfs*/ext4*/tmpfs*/lbs-xfs* to 'fstests' basename
- Fixes 'make: Nothing to be done for ci-test' error
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
scripts/ci.Makefile | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/scripts/ci.Makefile b/scripts/ci.Makefile
index 1c47c618..9d8151c3 100644
--- a/scripts/ci.Makefile
+++ b/scripts/ci.Makefile
@@ -10,8 +10,21 @@ ifneq (y,$(CONFIG_WORKFLOW_LINUX_PACKAGED))
ifeq ($(strip $(CI_WORKFLOW)),)
CI_WORKFLOW_BASENAME := $(shell basename $(CONFIG_BOOTLINUX_TREE) | sed 's/\.git$$//')
else
+# Map workflow names to their appropriate .ci basename
+ifeq ($(findstring xfs,$(CI_WORKFLOW)),xfs)
+CI_WORKFLOW_BASENAME := fstests
+else ifeq ($(findstring btrfs,$(CI_WORKFLOW)),btrfs)
+CI_WORKFLOW_BASENAME := fstests
+else ifeq ($(findstring ext4,$(CI_WORKFLOW)),ext4)
+CI_WORKFLOW_BASENAME := fstests
+else ifeq ($(findstring tmpfs,$(CI_WORKFLOW)),tmpfs)
+CI_WORKFLOW_BASENAME := fstests
+else ifeq ($(findstring lbs-xfs,$(CI_WORKFLOW)),lbs-xfs)
+CI_WORKFLOW_BASENAME := fstests
+else
CI_WORKFLOW_BASENAME := $(shell basename $(CI_WORKFLOW))
endif
+endif
ifneq ($(wildcard .ci/build-test/$(CI_WORKFLOW_BASENAME)),)
ifneq ($(wildcard .ci/test/$(CI_WORKFLOW_BASENAME)),)
--
2.50.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH RFC 7/8] fstests: fix ansible_host undefined error
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
` (5 preceding siblings ...)
2025-09-23 21:13 ` [PATCH RFC 6/8] scripts: fix CI_WORKFLOW basename mapping Daniel Gomez
@ 2025-09-23 21:13 ` Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 8/8] github: refactor into reusable actions and workflows Daniel Gomez
2025-09-24 1:14 ` [PATCH RFC 0/8] Refactor CI Luis Chamberlain
8 siblings, 0 replies; 10+ messages in thread
From: Daniel Gomez @ 2025-09-23 21:13 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez
From: Daniel Gomez <da.gomez@samsung.com>
Fix Ansible template error: 'ansible_host' is undefined
- Replace ansible_host with (ansible_host | default(inventory_hostname))
- Issue occurs when hosts file uses simple hostname entries without
explicit ansible_host definitions
- Template now falls back to inventory_hostname when ansible_host
unavailable
Root cause: fstests role assumed ansible_host always available, but
libvirt/simple inventory setups only define inventory_hostname. The
undefined template variable caused task failure and subsequent hang.
Generated-by: Claude AI
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
playbooks/roles/fstests/tasks/main.yml | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/playbooks/roles/fstests/tasks/main.yml b/playbooks/roles/fstests/tasks/main.yml
index 3a57a73b..f12bfdae 100644
--- a/playbooks/roles/fstests/tasks/main.yml
+++ b/playbooks/roles/fstests/tasks/main.yml
@@ -730,7 +730,7 @@
- name: Set the export volname prefix
tags: vars
ansible.builtin.set_fact:
- volname_prefix: "{{ ansible_host | regex_replace(kdevops_host_prefix + '-') }}-fs"
+ volname_prefix: "{{ (ansible_host | default(inventory_hostname)) | regex_replace(kdevops_host_prefix + '-') }}-fs"
when:
- fstests_fstyp == "nfs" or fstests_fstyp == "cifs"
@@ -919,7 +919,7 @@
- name: Verify section name {{ fstests_section }} exists on fstests config file
vars:
- fstests_section: "{{ ansible_host | regex_replace(kdevops_host_prefix + '-') | regex_replace('-dev') | regex_replace('-', '_') }}"
+ fstests_section: "{{ (ansible_host | default(inventory_hostname)) | regex_replace(kdevops_host_prefix + '-') | regex_replace('-dev') | regex_replace('-', '_') }}"
tags: ["oscheck", "fstests", "run_tests", "section"]
become: true
become_flags: "su - -c"
@@ -970,7 +970,7 @@
- name: Check and verify fstests dependencies are met prior to running fstests
vars:
- fstests_section: "{{ ansible_host | regex_replace(kdevops_host_prefix + '-') | regex_replace('-dev') | regex_replace('-', '_') }}"
+ fstests_section: "{{ (ansible_host | default(inventory_hostname)) | regex_replace(kdevops_host_prefix + '-') | regex_replace('-dev') | regex_replace('-', '_') }}"
tags: ["oscheck", "fstests", "run_tests"]
become: true
become_flags: "su - -c"
@@ -1194,7 +1194,7 @@
- name: Run oscheck-get-failures.sh to get list of known failed tests
vars:
- fstests_section: "{{ ansible_host | regex_replace(kdevops_host_prefix + '-') | regex_replace('-dev') | regex_replace('-', '_') }}"
+ fstests_section: "{{ (ansible_host | default(inventory_hostname)) | regex_replace(kdevops_host_prefix + '-') | regex_replace('-dev') | regex_replace('-', '_') }}"
tags: ["oscheck", "fstests", "run_tests"]
ansible.builtin.command: "{{ kdevops_data }}/workflows/fstests/scripts/oscheck-get-failures.sh --test-section {{ fstests_section }}"
args:
@@ -1254,7 +1254,7 @@
- name: Run fstests using ./oscheck.sh --print-start --journal {{ fstests_journal }} --print-done --test-section {{ fstests_section }} {{ oscheck_extra_args }} {{
all_limit_tests }}
vars:
- fstests_section: "{{ ansible_host | regex_replace(kdevops_host_prefix + '-') | regex_replace('-dev') | regex_replace('-', '_') }}"
+ fstests_section: "{{ (ansible_host | default(inventory_hostname)) | regex_replace(kdevops_host_prefix + '-') | regex_replace('-dev') | regex_replace('-', '_') }}"
initial_baseline_expunges: "{{ fstests_initial_baseline_args | trim }}"
skip_tests_args: "{{ fstests_skip_tests_args | trim }}"
tags: ["oscheck", "fstests", "run_tests"]
--
2.50.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH RFC 8/8] github: refactor into reusable actions and workflows
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
` (6 preceding siblings ...)
2025-09-23 21:13 ` [PATCH RFC 7/8] fstests: fix ansible_host undefined error Daniel Gomez
@ 2025-09-23 21:13 ` Daniel Gomez
2025-09-24 1:14 ` [PATCH RFC 0/8] Refactor CI Luis Chamberlain
8 siblings, 0 replies; 10+ messages in thread
From: Daniel Gomez @ 2025-09-23 21:13 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez
From: Daniel Gomez <da.gomez@samsung.com>
Refactor the .github/ CI support into reusable actions and workflows.
This allows to use the new actions not only internally in kdevops but
also in other projects such linux-modules-kpd and similar. Keeping the
ci logic in one place and allowing other projects to get the latest
changes.
CI supports the following:
- Split workflow in setup, test, archive and cleanup actions
- Fixed kernel: v6.15 on linux tree. This is to speed up builds relying
on the same kernel. We can update when needed or timely basis
- Fixed workflows: blktests_nvme and xfs_reflink_4k. We want to validate
kdevops workflows and blktests with nvme section/profile has been added
on top of the current fstests/xfs reflink 4k
- 2 CI modes:
A. kdevops selftest/ci validation. A single test execution for the given
workflow (e.g. generic/003 for xfs). This has it's own commit format.
B. kdevops full workflow. This is to validate a specific kernel on
a given test suite. It's supported on manual trigger as well as on
schedule.
Add automatic runner assignment based on ci_workflow names to enable
parallel execution on dedicated runners:
- Parallelize GitHub workflow execution on dedicated runners based on
CI_WORKFLOW names (using tags).
E.g.:
* blktests workflows → [self-hosted, Linux, X64, blktests]
* xfs workflows → [self-hosted, Linux, X64, xfs]
- Set timeout 120 minutes for CI B mode.
- Disable fail-fast to matrix strategy in both push.yml and schedule.yml
workflows to prevent cancellation of successful jobs when other matrix
jobs fail. Example: this allows blktests_nvme to continue running even
if xfs_reflink_4k fails
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
.github/actions/archive/action.yml | 28 ++++
.github/actions/cleanup/action.yml | 24 ++++
.github/actions/setup/action.yml | 187 ++++++++++++++++++++++++
.github/actions/test/action.yml | 213 +++++++++++++++++++++++++++
.github/workflows/destroy.yml | 93 ++++++++++++
.github/workflows/docker-tests.yml | 2 +-
.github/workflows/fstests.yml | 98 -------------
.github/workflows/main.yml | 97 +++++++++++++
.github/workflows/manual.yml | 122 ++++++++++++++++
.github/workflows/push.yml | 29 ++++
.github/workflows/schedule.yml | 54 +++++++
scripts/generate_ci_commit_message.sh | 263 ++++++++++++++++++++++++++++++++++
scripts/github_output.sh | 9 ++
scripts/korg-releases.py | 94 ++++++++++++
14 files changed, 1214 insertions(+), 99 deletions(-)
diff --git a/.github/actions/archive/action.yml b/.github/actions/archive/action.yml
new file mode 100644
index 00000000..6efe20fc
--- /dev/null
+++ b/.github/actions/archive/action.yml
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Archive results
+description: Archive kdevops results in https://github.com/linux-kdevops/kdevops-results-archive.git
+inputs:
+ ci_workflow:
+ required: false
+ type: string
+ default: 'demo'
+ dir:
+ description: 'Directory'
+ required: true
+ default: 'workdir'
+
+runs:
+ using: "composite"
+ steps:
+ - name: Get systemd journal files
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ make journal-dump
+
+ - name: Build our kdevops archive results
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ make ci-archive CI_WORKFLOW="${{ inputs.ci_workflow }}"
diff --git a/.github/actions/cleanup/action.yml b/.github/actions/cleanup/action.yml
new file mode 100644
index 00000000..18a8656a
--- /dev/null
+++ b/.github/actions/cleanup/action.yml
@@ -0,0 +1,24 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Cleanup kdevops VMs
+description: Destroy VMs and cleanup workspace
+
+inputs:
+ dir:
+ description: 'Directory'
+ required: true
+ default: 'workdir'
+
+runs:
+ using: "composite"
+ steps:
+ - name: Run kdevops make destroy
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ make destroy
+
+ - name: Remove working-directory
+ shell: bash
+ run: |
+ rm --recursive --force --verbose ${{ inputs.dir }}
diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml
new file mode 100644
index 00000000..407c8ac3
--- /dev/null
+++ b/.github/actions/setup/action.yml
@@ -0,0 +1,187 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Setup kdevops
+description: Setup kdevops workspace
+
+inputs:
+ ci_workflow:
+ required: false
+ type: string
+ default: 'demo'
+ dir:
+ description: 'Directory'
+ required: true
+ default: 'workdir'
+ kernel_tree:
+ required: false
+ type: string
+ default: 'linux'
+ kernel_ref:
+ required: false
+ type: string
+ default: 'master'
+
+runs:
+ using: "composite"
+ steps:
+ - name: Create workspace directory
+ shell: bash
+ run: |
+ set -euxo pipefail
+ rm --recursive --force --verbose ${{ inputs.dir }}
+ mkdir --parent --verbose ${{ inputs.dir }}
+
+ - name: Configure git
+ shell: bash
+ run: |
+ set -euxo pipefail
+ git config --global --add safe.directory '*'
+ git config --global user.name "kdevops"
+ git config --global user.email "kdevops@lists.linux.dev"
+
+ - name: Checkout kdevops
+ working-directory: ${{ inputs.dir }}
+ shell: bash
+ run: |
+ set -euxo pipefail
+ git clone https://github.com/dkruces/kdevops.git --branch main kdevops
+
+ - name: Checkout custom branch with delta on kdevops/linux
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ LINUX_TREE="/mirror/${{ inputs.kernel_tree }}.git"
+ LINUX_TREE_REF="${{ inputs.kernel_ref }}"
+ git clone $LINUX_TREE linux
+ cd linux
+ git checkout $LINUX_TREE_REF
+ git log -1
+
+ - name: Make sure our repo kdevops defconfig exists
+ id: defconfig
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ KDEVOPS_DEFCONFIG=${{ inputs.ci_workflow }}
+
+ if [[ ! -f defconfigs/$KDEVOPS_DEFCONFIG ]]; then
+ echo "Missing defconfig: defconfigs/$KDEVOPS_DEFCONFIG"
+ exit 1
+ fi
+
+ "${{ github.workspace }}/scripts/github_output.sh" \
+ KDEVOPS_DEFCONFIG "$KDEVOPS_DEFCONFIG"
+
+ - name: Initialize CI metadata for kdevops-results-archive for linux
+ id: metadata
+ working-directory: ${{ inputs.dir }}/kdevops/linux
+ shell: bash
+ run: |
+ set -euxo pipefail
+ echo "${{ inputs.kernel_tree }}" > ../ci.trigger
+ # Get the kdevops commit subject, not kernel commit
+ cd "${{ github.workspace }}" && git log -1 --pretty=format:"%s" > "${{ inputs.dir }}/kdevops/ci.subject"
+ cd - > /dev/null
+ echo "${{ inputs.kernel_ref }}" > ../ci.ref
+
+ RELEVANT_GIT_TAG=$(cat ../ci.ref)
+ RELEVANT_GIT_REF=$(git rev-parse --short=12 $RELEVANT_GIT_TAG)
+
+ "${{ github.workspace }}/scripts/github_output.sh" \
+ LINUX_GIT_REF "$RELEVANT_GIT_REF"
+ "${{ github.workspace }}/scripts/github_output.sh" \
+ LINUX_GIT_TAG "$RELEVANT_GIT_TAG"
+
+ # Start out pessimistic
+ echo "unknown" > ../ci.result
+ echo "Nothing to write home about." > ../ci.commit_extra
+
+ - name: Run kdevops make defconfig-repo
+ working-directory: ${{ inputs.dir }}/kdevops
+ env:
+ LINUX_GIT_TAG: ${{ steps.metadata.outputs.LINUX_GIT_TAG }}
+ LINUX_GIT_REF: ${{ steps.metadata.outputs.LINUX_GIT_REF }}
+ KDEVOPS_DEFCONFIG: ${{ steps.defconfig.outputs.KDEVOPS_DEFCONFIG }}
+ shell: bash
+ run: |
+ set -euxo pipefail
+ LINUX_TREE="/mirror/${{ inputs.kernel_tree }}.git"
+ LINUX_TREE_REF="$LINUX_GIT_TAG"
+
+ # We make the compromise here to use a relevant git tag for the
+ # host prefix so that folks can easily tell what exact kernel tree
+ # is being tested by using the relevant git ref. That is, if you
+ # pushed a tree with the .github/ directory as the top of the tree,
+ # that commit will not be used, we'll use the last one as that is
+ # the relevant git ref we want to annotate a test for.
+ #
+ # The compromise here is that we expect no two same identical tests
+ # on the same self-hosted server. We could extend this with something
+ # like github.run_id but its not yet clear if we can have kdevops
+ # hosts with a bundled prefix ID like that ref-runid-testname. Tests
+ # have been done with the full lenght sha1sum though and we know that
+ # does work.
+ KDEVOPS_HOSTS_PREFIX="kci-$LINUX_GIT_REF"
+
+ echo "Going to use defconfig-$KDEVOPS_DEFCONFIG"
+
+ echo "Linux tree: $LINUX_TREE"
+ echo "Linux trigger ref: $LINUX_TREE_REF"
+ echo "Linux tag: $LINUX_GIT_TAG"
+ echo "Runner ID: ${{ github.run_id }}"
+ echo "kdevops host prefix: $KDEVOPS_HOSTS_PREFIX"
+ echo "kdevops defconfig: defconfig-$KDEVOPS_DEFCONFIG"
+
+ # Customize KMOD_TIMEOUT when required
+ KMOD_TIMEOUT_ARG=
+ if [[ "$(hostname)" == *smc111* && \
+ "$KDEVOPS_DEFCONFIG" == "linux-modules-kpd" ]]; then
+ KMOD_TIMEOUT_ARG="KMOD_TIMEOUT=222"
+ fi
+
+ KDEVOPS_ARGS="\
+ KDEVOPS_HOSTS_PREFIX=$KDEVOPS_HOSTS_PREFIX \
+ LINUX_TREE=$LINUX_TREE \
+ LINUX_TREE_REF=$LINUX_TREE_REF \
+ ${KMOD_TIMEOUT_ARG} \
+ defconfig-$KDEVOPS_DEFCONFIG"
+ echo "Going to run:"
+ echo "make $KDEVOPS_ARGS"
+
+ make $KDEVOPS_ARGS
+ ./scripts/kconfig/merge_config.sh \
+ -n .config \
+ defconfigs/configs/diy.config \
+ defconfigs/configs/ci.config
+
+ - name: Run kdevops make
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ make -j$(nproc)
+
+ - name: Run kdevops make bringup
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ ls -ld linux
+ make destroy
+ make bringup
+
+ - name: Build linux and boot test nodes on test kernel
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ make linux
+
+ - name: Build required ci tests
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ make ci-build-test CI_WORKFLOW=${{ inputs.ci_workflow }}
diff --git a/.github/actions/test/action.yml b/.github/actions/test/action.yml
new file mode 100644
index 00000000..a9022b4c
--- /dev/null
+++ b/.github/actions/test/action.yml
@@ -0,0 +1,213 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Setup kdevops
+description: Setup kdevops workspace
+
+inputs:
+ dir:
+ description: 'Directory'
+ required: true
+ default: 'workdir'
+ ci_workflow:
+ required: false
+ type: string
+ default: 'demo'
+ test_mode:
+ description: 'Testing mode'
+ required: false
+ default: 'full-testing'
+ tests:
+ description: 'Custom test to run (for kdevops-validation mode only)'
+ required: false
+ default: ''
+
+runs:
+ using: "composite"
+ steps:
+ - name: Run CI tests
+ id: ci_test
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+
+ # Create start time for duration calculation
+ echo "$(date +%s)" > ci.start_time
+
+ # Handle test mode and tests parameters
+ if [[ -n "${{ inputs.tests }}" ]]; then
+ # Custom test specified for kdevops-validation mode
+ TESTS="${{ inputs.tests }}"
+ elif [[ "${{ inputs.test_mode }}" == "kdevops-validation" ]]; then
+ # Auto-assign appropriate test based on workflow
+ case "${{ inputs.ci_workflow }}" in
+ *fstests*|*xfs*|*btrfs*|*ext4*|*tmpfs*)
+ TESTS="generic/003"
+ ;;
+ blktests*)
+ TESTS="block/003"
+ ;;
+ selftests*|*modules*|*kmod*|*firmware*|*mm*)
+ TESTS="kmod/test_001"
+ ;;
+ *)
+ TESTS="generic/003"
+ ;;
+ esac
+ else
+ # Full testing mode - no TESTS variable
+ TESTS=""
+ fi
+
+ # Export TESTS for current step and save to GitHub output for later steps
+ export TESTS="$TESTS"
+ "${{ github.workspace }}/scripts/github_output.sh" TESTS "$TESTS"
+
+ make ci-test CI_WORKFLOW="${{ inputs.ci_workflow }}"
+
+ - name: Generate workflow results path
+ id: setpath
+ shell: bash
+ run: |
+ set -euxo pipefail
+
+ case "${{ inputs.ci_workflow }}" in
+ blktests*) wpath="workflows/blktests" ;;
+ *btrfs*) wpath="workflows/fstests" ;;
+ *ext4*) wpath="workflows/fstests" ;;
+ tmpfs*) wpath="workflows/fstests" ;;
+ *xfs*) wpath="workflows/fstests" ;;
+ *) wpath="workflows/selftests" ;;
+ esac
+
+ "${{ github.workspace }}/scripts/github_output.sh" wpath "$wpath"
+
+ - name: Generate CI commit info with workflow-specific logic
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+
+ wpath="${{ steps.setpath.outputs.wpath }}"
+
+ # Workflow-specific result collection
+ case "${{ inputs.ci_workflow }}" in
+ *xfs*|*btrfs*|*ext4*|tmpfs*|*fstests*)
+ # fstests workflows: Use xunit_results.txt for rich summary
+ echo "Collecting fstests results..."
+ if find "$wpath/results/last-run" -name "xunit_results.txt" -type f -exec cat {} \; > ci.commit_extra 2>/dev/null; then
+ echo "Found xunit_results.txt"
+ else
+ echo "No xunit_results.txt found, using fallback..."
+ echo "Kernel tests results:" > ci.commit_extra
+ find "$wpath/results/last-run/" -name '*.dmesg.log' -exec tail -n 1 {} + >> ci.commit_extra 2>/dev/null || true
+ fi
+
+ # fstests success detection
+ if ! grep -E "failures, [1-9]|errors, [1-9]" ci.commit_extra >/dev/null 2>&1; then
+ echo "ok" > ci.result
+ else
+ echo "not ok" > ci.result
+ fi
+ ;;
+ blktests*)
+ # blktests workflows: Collect individual test results
+
+ echo "Kernel tests results:" > ci.commit_extra
+
+ # Collect test results from last-run directory
+ if [ -d "$wpath/results/last-run" ]; then
+ echo -e "\nBlktests summary:" >> ci.commit_extra
+
+ # Count total tests by unique test names (not file extensions)
+ # Get base test names by removing extensions and extracting just the test name
+ test_names=$(find "$wpath/results/last-run" -type f -name "*" | \
+ grep -E '/[^/]*$' | \
+ sed 's|.*/||' | \
+ sed 's/\.\(full\|out\|runtime\|dmesg\)$//' | \
+ sort -u)
+ total_tests=$(echo "$test_names" | grep -v '^$' | wc -l)
+
+ bad_files=$(find "$wpath/results/last-run" -name "*.out.bad" -type f)
+ if [[ -n "$bad_files" ]]; then
+ failed_tests=$(echo "$bad_files" | wc -l)
+ else
+ failed_tests=0
+ fi
+
+ passed_tests=$((total_tests - failed_tests))
+
+ echo "Tests run: $total_tests, Passed: $passed_tests, Failed: $failed_tests" >> ci.commit_extra
+
+ # List failed tests if any
+ if [ $failed_tests -gt 0 ]; then
+ echo -e "\nFailed tests:" >> ci.commit_extra
+ find "$wpath/results/last-run" -name "*.out.bad" -type f | sed 's|.*/\([^/]*\)\.out\.bad$|\1|' | sort >> ci.commit_extra 2>/dev/null || true
+ fi
+
+ # Show sample test status files for passed tests
+ if [ $passed_tests -gt 0 ]; then
+ echo -e "\nSample passed test:" >> ci.commit_extra
+ sample_files=$(find "$wpath/results/last-run" -type f -name "*" ! -name "*.out.bad" ! -name "*.dmesg")
+ sample_file=$(echo "$sample_files" | head -1)
+ if [ -n "$sample_file" ] && [ -f "$sample_file" ]; then
+ cat "$sample_file" >> ci.commit_extra || echo "Failed to read sample file" >> ci.commit_extra
+ else
+ echo "No valid sample file found" >> ci.commit_extra
+ fi
+ fi
+ else
+ echo -e "\nNo blktests results found in $wpath/results/last-run" >> ci.commit_extra
+ fi
+
+ # blktests success detection - look for .out.bad files (failures)
+ bad_check=$(find "$wpath/results/last-run" -name "*.out.bad" -type f | head -1)
+
+ if [ -n "$bad_check" ]; then
+ echo "not ok" > ci.result
+ else
+ echo "ok" > ci.result
+ fi
+ ;;
+ *selftests*|*modules*|*mm*|*firmware*)
+ # selftests workflows: Use userspace.log
+ echo "Kernel tests results:" > ci.commit_extra
+ find "$wpath/results/last-run/" -name '*.userspace.log' -exec cat {} \; >> ci.commit_extra 2>/dev/null || true
+
+ # selftests success detection
+ if grep -q "passed\|PASS" ci.commit_extra && ! grep -q "FAIL\|failed\|ERROR" ci.commit_extra; then
+ echo "ok" > ci.result
+ else
+ echo "not ok" > ci.result
+ fi
+ ;;
+ *)
+ # Default/unknown workflows: Generic approach
+ echo "Kernel tests results:" > ci.commit_extra
+ find "$wpath/results/last-run/" -name '*.dmesg.log' -exec tail -n 1 {} + >> ci.commit_extra 2>/dev/null || true
+ echo -e "\n\nUserspace test results:" >> ci.commit_extra
+ find "$wpath/results/last-run/" -name '*.userspace.log' -exec tail -n 1 {} + >> ci.commit_extra 2>/dev/null || true
+
+ if grep -i -q "fail" ci.commit_extra; then
+ echo "not ok" > ci.result
+ else
+ echo "ok" > ci.result
+ fi
+ ;;
+ esac
+
+ # Set environment variables for the commit message script
+ export CI_WORKFLOW="${{ inputs.ci_workflow }}"
+ export KERNEL_TREE="${{ inputs.kernel_tree || 'linux' }}"
+
+ # Get TESTS from GitHub output (determines kdevops validation vs full testing)
+ TESTS="${{ steps.ci_test.outputs.TESTS }}"
+ if [[ -n "${TESTS:-}" ]]; then
+ export TESTS="$TESTS"
+ fi
+
+ # Generate enhanced commit message using our script
+ ./scripts/generate_ci_commit_message.sh > ci.commit_message_enhanced
+
+ # Keep the original ci.commit_extra for backward compatibility
+ # The archive action will use ci.commit_message_enhanced if available
diff --git a/.github/workflows/destroy.yml b/.github/workflows/destroy.yml
new file mode 100644
index 00000000..6689452b
--- /dev/null
+++ b/.github/workflows/destroy.yml
@@ -0,0 +1,93 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Run kdevops Destroyer - Manual
+
+on:
+ workflow_dispatch:
+ inputs:
+ ci_workflow:
+ description: "CI Workflow"
+ required: true
+ default: 'blktests_nvme'
+ type: choice
+ options:
+ - blktests
+ - blktests_block
+ - blktests_loop
+ - blktests_meta
+ - blktests_nbd
+ - blktests_nvme
+ - blktests_nvmemp
+ - blktests_scsi
+ - blktests_srp
+ - blktests_zbd
+ - lbs-xfs
+ - lbs-xfs-bdev-large-nvme
+ - lbs-xfs-bdev-nvme
+ - lbs-xfs-small
+ - xfs
+ - xfs_crc
+ - xfs_crc_logdev
+ - xfs_crc_rtdev
+ - xfs_crc_rtdev_extsize_28k
+ - xfs_crc_rtdev_extsize_64k
+ - xfs_nocrc
+ - xfs_nocrc_16k_4ks
+ - xfs_nocrc_1k
+ - xfs_nocrc_2k
+ - xfs_nocrc_32_4ks
+ - xfs_nocrc_4k
+ - xfs_nocrc_512
+ - xfs_nocrc_64_4ks
+ - xfs_nocrc_8k_4ks
+ - xfs_nocrc_lbs
+ - xfs_reflink
+ - xfs_reflink_1024
+ - xfs_reflink_16k_4ks
+ - xfs_reflink_2k
+ - xfs_reflink_32k_4ks
+ - xfs_reflink_4k
+ - xfs_reflink_4k_8ks
+ - xfs_reflink_64k_4ks
+ - xfs_reflink_dir_bsize_8k
+ - xfs_reflink_lbs
+ - xfs_reflink_logdev
+ - xfs_reflink_normapbt
+ - xfs_reflink_nrext64
+ - xfs_reflink_stripe_len
+ - xfs-soak
+ host_prefix:
+ description: "Host prefix"
+ required: true
+ default: 'debian13'
+ type: string
+
+jobs:
+ destroyer:
+ name: Destroy guests
+ runs-on: [self-hosted]
+ steps:
+ - name: Create workdir
+ run: |
+ rm --recursive --force --verbose ${{ inputs.ci_workflow }}
+ mkdir --parent --verbose ${{ inputs.ci_workflow }}
+
+ - name: Checkout kdevops
+ run: |
+ cd ${{ inputs.ci_workflow }}
+ git clone https://github.com/dkruces/kdevops.git --branch ci-workflow kdevops
+
+ - name: Run kdevops make defconfig-repo
+ run: |
+ set -euxo pipefail
+
+ sudo virsh list --all
+
+ cd ${{ inputs.ci_workflow }}/kdevops
+ make \
+ KDEVOPS_HOSTS_PREFIX="${{ inputs.host_prefix }}" \
+ defconfig-${{ inputs.ci_workflow }}
+
+ make destroy
+
+ sudo virsh list --all
diff --git a/.github/workflows/docker-tests.yml b/.github/workflows/docker-tests.yml
index c0e0d03d..33d67f09 100644
--- a/.github/workflows/docker-tests.yml
+++ b/.github/workflows/docker-tests.yml
@@ -1,4 +1,4 @@
-name: Setup kdevops
+name: kdevops Docker Tests
on:
push:
diff --git a/.github/workflows/fstests.yml b/.github/workflows/fstests.yml
deleted file mode 100644
index e13978c7..00000000
--- a/.github/workflows/fstests.yml
+++ /dev/null
@@ -1,98 +0,0 @@
-name: Run kdevops on self-hosted runner
-
-on:
- push:
- branches:
- - '**'
- pull_request:
- branches:
- - '**'
- workflow_dispatch: # Add this for manual triggering of the workflow
-
-jobs:
- run-kdevops:
- name: Run kdevops CI
- runs-on: [self-hosted, Linux, X64]
- steps:
- - name: Checkout repository
- uses: actions/checkout@v4
-
- - name: Set CI metadata for kdevops-results-archive
- run: |
- echo "$(basename ${{ github.repository }})" > ci.trigger
- git log -1 --pretty=format:"%s" > ci.subject
- # Start out pessimistic
- echo "not ok" > ci.result
- echo "Nothing to write home about." > ci.commit_extra
-
- - name: Set kdevops path
- run: echo "KDEVOPS_PATH=$GITHUB_WORKSPACE" >> $GITHUB_ENV
-
- - name: Configure git
- run: |
- git config --global --add safe.directory '*'
- git config --global user.name "kdevops"
- git config --global user.email "kdevops@lists.linux.dev"
-
- - name: Run kdevops make defconfig-repo
- run: |
- KDEVOPS_TREE_REF="${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}"
- SHORT_PREFIX="$(echo ${KDEVOPS_TREE_REF:0:12})"
- make KDEVOPS_HOSTS_PREFIX="$SHORT_PREFIX" \
- ANSIBLE_CFG_CALLBACK_PLUGIN="debug" \
- LINUX_TREE_REF="v6.15" \
- defconfig-xfs_reflink_4k
-
- - name: Run kdevops make
- run: |
- make V=1 -j$(nproc)
-
- - name: Run kdevops make bringup
- run: |
- make V=1 bringup
-
- - name: Build linux and boot test nodes on test kernel
- run: |
- make V=1 linux
-
- - name: Build fstests
- run: |
- make V=1 fstests
-
- - name: Run just one fstest to verify we tests and test collection works
- run: |
- make V=1 fstests-baseline TESTS=generic/003
- echo "ok" > ci.result
- find workflows/fstests/results/last-run -name xunit_results.txt -type f -exec cat {} \; > ci.commit_extra || true
- if ! grep -E "failures, [1-9]|errors, [1-9]" ci.commit_extra; then
- echo "ok" > ci.result
- fi
-
- - name: Get systemd journal files
- if: always() # This ensures the step runs even if previous steps failed
- run: |
- make V=1 journal-dump
-
- - name: Start SSH Agent
- if: always() # Ensure this step runs even if previous steps failed
- uses: webfactory/ssh-agent@v0.9.0
- with:
- ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
-
- - name: Build our kdevops archive results
- if: always() # This ensures the step runs even if previous steps failed
- run: |
- make V=1 ci-archive
-
- - name: Upload our kdevops results archive
- if: always() # This ensures the step runs even if previous steps failed
- uses: actions/upload-artifact@v4
- with:
- name: kdevops-ci-results
- path: ${{ env.KDEVOPS_PATH }}/archive/*.zip
-
- # Ensure make destroy always runs, even on failure
- - name: Run kdevops make destroy
- if: always() # This ensures the step runs even if previous steps failed
- run: |
- make V=1 destroy
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
new file mode 100644
index 00000000..35423403
--- /dev/null
+++ b/.github/workflows/main.yml
@@ -0,0 +1,97 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Run kdevops CI Workflow - Reusable
+
+on:
+ workflow_call:
+ inputs:
+ ci_workflow:
+ description: "CI Workflow"
+ required: true
+ default: 'blktests_nvme'
+ type: string
+ kernel_ref:
+ description: "Linux tree git reference (branch/tag/commit-id)"
+ required: true
+ default: 'master'
+ type: string
+ kernel_tree:
+ description: "Linux kernel tree to use"
+ required: true
+ default: 'linux'
+ type: string
+ test_mode:
+ description: 'Testing mode'
+ required: false
+ default: 'full-testing'
+ type: string
+ tests:
+ description: 'Custom test to run (for kdevops-validation mode only)'
+ required: false
+ default: ''
+ type: string
+
+jobs:
+ setup:
+ name: Setup kdevops workspace
+ runs-on: ${{ contains(inputs.ci_workflow, 'blktests') && fromJSON('["self-hosted", "Linux", "X64", "blktests"]') || contains(inputs.ci_workflow, 'tmpfs') && fromJSON('["self-hosted", "Linux", "X64", "tmpfs"]') || contains(inputs.ci_workflow, 'xfs') && fromJSON('["self-hosted", "Linux", "X64", "xfs"]') || contains(inputs.ci_workflow, 'btrfs') && fromJSON('["self-hosted", "Linux", "X64", "btrfs"]') || contains(inputs.ci_workflow, 'ext4') && fromJSON('["self-hosted", "Linux", "X64", "ext4"]') || contains(inputs.ci_workflow, 'selftests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'seltests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'modules') && fromJSON('["self-hosted", "Linux", "X64", "modules"]') || contains(inputs.ci_workflow, 'firmware') && fromJSON('["self-hosted", "Linux", "X64", "firmware"]') || contains(inputs.ci_workflow, 'mm') && fromJSON('["self-hosted", "Linux", "X64", "mm"]') || fromJSON('["self-hosted"]') }}
+ steps:
+ - name: Checkout dkruces/kdevops
+ uses: actions/checkout@v4
+ with:
+ clean: false
+
+ - name: kdevops setup
+ uses: ./.github/actions/setup
+ with:
+ ci_workflow: ${{ inputs.ci_workflow }}
+ dir: ${{ inputs.ci_workflow }}
+ kernel_ref: ${{ inputs.kernel_ref }}
+ kernel_tree: ${{ inputs.kernel_tree }}
+
+ test:
+ name: Run kdevops ci-test
+ runs-on: ${{ contains(inputs.ci_workflow, 'blktests') && fromJSON('["self-hosted", "Linux", "X64", "blktests"]') || contains(inputs.ci_workflow, 'tmpfs') && fromJSON('["self-hosted", "Linux", "X64", "tmpfs"]') || contains(inputs.ci_workflow, 'xfs') && fromJSON('["self-hosted", "Linux", "X64", "xfs"]') || contains(inputs.ci_workflow, 'btrfs') && fromJSON('["self-hosted", "Linux", "X64", "btrfs"]') || contains(inputs.ci_workflow, 'ext4') && fromJSON('["self-hosted", "Linux", "X64", "ext4"]') || contains(inputs.ci_workflow, 'selftests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'seltests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'modules') && fromJSON('["self-hosted", "Linux", "X64", "modules"]') || contains(inputs.ci_workflow, 'firmware') && fromJSON('["self-hosted", "Linux", "X64", "firmware"]') || contains(inputs.ci_workflow, 'mm') && fromJSON('["self-hosted", "Linux", "X64", "mm"]') || fromJSON('["self-hosted"]') }}
+ needs: [setup]
+ timeout-minutes: 120
+ steps:
+ - name: kdevops ci-test
+ uses: ./.github/actions/test
+ with:
+ ci_workflow: ${{ inputs.ci_workflow }}
+ dir: ${{ inputs.ci_workflow }}
+ test_mode: ${{ inputs.test_mode }}
+ tests: ${{ inputs.tests }}
+
+ archive:
+ name: Archive kdevops
+ runs-on: ${{ contains(inputs.ci_workflow, 'blktests') && fromJSON('["self-hosted", "Linux", "X64", "blktests"]') || contains(inputs.ci_workflow, 'tmpfs') && fromJSON('["self-hosted", "Linux", "X64", "tmpfs"]') || contains(inputs.ci_workflow, 'xfs') && fromJSON('["self-hosted", "Linux", "X64", "xfs"]') || contains(inputs.ci_workflow, 'btrfs') && fromJSON('["self-hosted", "Linux", "X64", "btrfs"]') || contains(inputs.ci_workflow, 'ext4') && fromJSON('["self-hosted", "Linux", "X64", "ext4"]') || contains(inputs.ci_workflow, 'selftests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'seltests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'modules') && fromJSON('["self-hosted", "Linux", "X64", "modules"]') || contains(inputs.ci_workflow, 'firmware') && fromJSON('["self-hosted", "Linux", "X64", "firmware"]') || contains(inputs.ci_workflow, 'mm') && fromJSON('["self-hosted", "Linux", "X64", "mm"]') || fromJSON('["self-hosted"]') }}
+ needs: [setup, test]
+ steps:
+ - name: Start SSH Agent
+ uses: webfactory/ssh-agent@v0.9.0
+ with:
+ ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
+
+ - name: Archive ci-test results
+ uses: ./.github/actions/archive
+ with:
+ ci_workflow: ${{ inputs.ci_workflow }}
+ dir: ${{ inputs.ci_workflow }}
+
+ - name: Upload our kdevops results archive
+ uses: actions/upload-artifact@v4
+ with:
+ name: kdevops-ci-results
+ path: ${{ inputs.ci_workflow }}/kdevops/archive/*.zip
+
+ cleanup:
+ name: Cleanup kdevops workspace
+ runs-on: ${{ contains(inputs.ci_workflow, 'blktests') && fromJSON('["self-hosted", "Linux", "X64", "blktests"]') || contains(inputs.ci_workflow, 'tmpfs') && fromJSON('["self-hosted", "Linux", "X64", "tmpfs"]') || contains(inputs.ci_workflow, 'xfs') && fromJSON('["self-hosted", "Linux", "X64", "xfs"]') || contains(inputs.ci_workflow, 'btrfs') && fromJSON('["self-hosted", "Linux", "X64", "btrfs"]') || contains(inputs.ci_workflow, 'ext4') && fromJSON('["self-hosted", "Linux", "X64", "ext4"]') || contains(inputs.ci_workflow, 'selftests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'seltests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'modules') && fromJSON('["self-hosted", "Linux", "X64", "modules"]') || contains(inputs.ci_workflow, 'firmware') && fromJSON('["self-hosted", "Linux", "X64", "firmware"]') || contains(inputs.ci_workflow, 'mm') && fromJSON('["self-hosted", "Linux", "X64", "mm"]') || fromJSON('["self-hosted"]') }}
+ needs: [setup, test, archive]
+ if: always()
+ steps:
+ - name: kdevops cleanup
+ uses: ./.github/actions/cleanup
+ with:
+ dir: ${{ inputs.ci_workflow }}
diff --git a/.github/workflows/manual.yml b/.github/workflows/manual.yml
new file mode 100644
index 00000000..30a035c8
--- /dev/null
+++ b/.github/workflows/manual.yml
@@ -0,0 +1,122 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Run kdevops CI Workflow - Manual
+
+on:
+ workflow_dispatch:
+ inputs:
+ ci_workflow:
+ description: "CI Workflow"
+ required: true
+ default: 'blktests_nvme'
+ type: choice
+ options:
+ - blktests
+ - blktests_block
+ - blktests_loop
+ - blktests_meta
+ - blktests_nbd
+ - blktests_nvme
+ - blktests_nvmemp
+ - blktests_scsi
+ - blktests_srp
+ - blktests_zbd
+ - lbs-xfs
+ - lbs-xfs-bdev-large-nvme
+ - lbs-xfs-bdev-nvme
+ - lbs-xfs-small
+ - xfs
+ - xfs_crc
+ - xfs_crc_logdev
+ - xfs_crc_rtdev
+ - xfs_crc_rtdev_extsize_28k
+ - xfs_crc_rtdev_extsize_64k
+ - xfs_nocrc
+ - xfs_nocrc_16k_4ks
+ - xfs_nocrc_1k
+ - xfs_nocrc_2k
+ - xfs_nocrc_32_4ks
+ - xfs_nocrc_4k
+ - xfs_nocrc_512
+ - xfs_nocrc_64_4ks
+ - xfs_nocrc_8k_4ks
+ - xfs_nocrc_lbs
+ - xfs_reflink
+ - xfs_reflink_1024
+ - xfs_reflink_16k_4ks
+ - xfs_reflink_2k
+ - xfs_reflink_32k_4ks
+ - xfs_reflink_4k
+ - xfs_reflink_4k_8ks
+ - xfs_reflink_64k_4ks
+ - xfs_reflink_dir_bsize_8k
+ - xfs_reflink_lbs
+ - xfs_reflink_logdev
+ - xfs_reflink_normapbt
+ - xfs_reflink_nrext64
+ - xfs_reflink_stripe_len
+ - xfs-soak
+ kernel_tree:
+ description: "Linux kernel tree to use"
+ required: true
+ default: 'linux'
+ type: choice
+ options:
+ - linux
+ - linux-next
+ - linux-stable
+ kernel_ref:
+ description: "Linux tree git reference (branch/tag/commit-id)"
+ required: true
+ default: 'master'
+ type: string
+ test_mode:
+ description: 'Testing mode'
+ required: false
+ default: 'full-testing'
+ type: choice
+ options:
+ - 'full-testing' # Complete test suite
+ - 'kdevops-validation' # Single test for framework validation
+ tests:
+ description: 'Custom test to run (for kdevops-validation mode only)'
+ required: false
+ type: string
+ default: ''
+
+jobs:
+ check_ref:
+ name: Check Linux kernel Git Reference
+ runs-on: [self-hosted]
+ steps:
+ - name: Checkout repo
+ uses: actions/checkout@v4
+
+ - name: Check kernel_ref exists
+ id: check_kernel_ref
+ run: |
+ set -euxo pipefail
+
+ ref="${{ github.event.inputs.kernel_ref }}"
+ tree="${{ github.event.inputs.kernel_tree }}"
+ mirror="/mirror/${tree}.git"
+ ls_remote="$(git ls-remote "$mirror" "refs/*/${ref}" || true)"
+ contains_branch="$(git -C "$mirror" branch --contains "${ref}" || true)"
+ contains_tag="$(git -C "$mirror" branch --contains "${ref}" || true)"
+
+ if [ -z "$ls_remote" ] && [ -z "$contains_branch" ] && [ -z "$contains_tag" ]; then
+ echo "Linux kernel ${ref} does not exist."
+ exit 1
+ fi
+
+ manual:
+ name: Manual kdevops CI
+ needs: [check_ref]
+ uses: ./.github/workflows/main.yml
+ secrets: inherit
+ with:
+ ci_workflow: ${{ inputs.ci_workflow }}
+ kernel_ref: ${{ inputs.kernel_ref }}
+ kernel_tree: ${{ inputs.kernel_tree }}
+ test_mode: ${{ inputs.test_mode }}
+ tests: ${{ inputs.tests }}
diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml
new file mode 100644
index 00000000..561e873e
--- /dev/null
+++ b/.github/workflows/push.yml
@@ -0,0 +1,29 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Run kdevops CI Workflow - Push/PR
+
+on:
+ push:
+ branches:
+ - '**'
+ pull_request:
+ branches:
+ - '**'
+
+jobs:
+ push-pr:
+ name: Push/PR kdevops CI
+ uses: ./.github/workflows/main.yml
+ secrets: inherit
+ strategy:
+ fail-fast: false # Don't cancel other jobs if one fails
+ matrix:
+ ci_workflow:
+ - blktests_nvme
+ - xfs_reflink_4k
+ with:
+ ci_workflow: ${{ matrix.ci_workflow }}
+ kernel_ref: 'v6.15'
+ kernel_tree: 'linux'
+ test_mode: 'kdevops-validation'
+ tests: ''
diff --git a/.github/workflows/schedule.yml b/.github/workflows/schedule.yml
new file mode 100644
index 00000000..8f4589b4
--- /dev/null
+++ b/.github/workflows/schedule.yml
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Run kdevops CI Workflow - Schedule
+
+on:
+ schedule:
+ - cron: '0 14 * * *'
+
+ workflow_dispatch:
+
+jobs:
+ check_ref:
+ name: Check Linux kernel Git Reference
+ outputs:
+ kernel_ref: ${{ steps.check_kernel_ref.outputs.kernel_ref }}
+ kernel_tree: ${{ steps.check_kernel_ref.outputs.kernel_tree }}
+ runs-on: [self-hosted]
+ steps:
+ - name: Checkout repo
+ uses: actions/checkout@v4
+
+ - name: Generate kernel_ref
+ id: check_kernel_ref
+ run: |
+ set -euxo pipefail
+ day_of_week=$(date +%u)
+
+ kernel_ref=$(./scripts/korg-releases.py --moniker linux-next)
+ kernel_tree=linux-next
+ if [ "$day_of_week" -eq 1 ]; then
+ kernel_ref=$(./scripts/korg-releases.py --moniker mainline)
+ kernel_tree=linux
+ fi
+
+ "${{ github.workspace }}/scripts/github_output.sh" kernel_ref "$kernel_ref"
+ "${{ github.workspace }}/scripts/github_output.sh" kernel_tree "$kernel_tree"
+
+ schedule:
+ name: Scheduled kdevops CI
+ needs: [check_ref]
+ uses: ./.github/workflows/main.yml
+ secrets: inherit
+ strategy:
+ fail-fast: false # Don't cancel other jobs if one fails
+ matrix:
+ ci_workflow:
+ - blktests
+ - linux-mm-kpd
+ - linux-modules-kpd
+ - tmpfs
+ with:
+ ci_workflow: ${{ matrix.ci_workflow }}
+ kernel_ref: ${{ needs.check_ref.outputs.kernel_ref }}
+ kernel_tree: ${{ needs.check_ref.outputs.kernel_tree }}
diff --git a/scripts/generate_ci_commit_message.sh b/scripts/generate_ci_commit_message.sh
new file mode 100755
index 00000000..8be2c0b3
--- /dev/null
+++ b/scripts/generate_ci_commit_message.sh
@@ -0,0 +1,263 @@
+#!/bin/bash
+# SPDX-License-Identifier: copyleft-next-0.3.1
+
+# Generate enhanced CI commit message for kdevops-results-archive
+# This script creates commit messages following the kdevops CI format specification
+
+set -euo pipefail
+
+# Default values
+CI_WORKFLOW="${CI_WORKFLOW:-demo}"
+KERNEL_TREE="${KERNEL_TREE:-linux}"
+TESTS_PARAM="${TESTS:-${LIMIT_TESTS:-}}"
+DURATION="${DURATION:-unknown}"
+
+# Input files (expected to be created by GitHub Actions)
+CI_COMMIT_EXTRA="${CI_COMMIT_EXTRA:-ci.commit_extra}"
+CI_RESULT="${CI_RESULT:-ci.result}"
+CI_START_TIME="${CI_START_TIME:-ci.start_time}"
+CI_REF="${CI_REF:-ci.ref}"
+CI_TRIGGER="${CI_TRIGGER:-ci.trigger}"
+
+# Helper function to wrap long commit subjects at 72 chars
+wrap_commit_subject() {
+ local subject="$1"
+ if [ ${#subject} -le 68 ]; then # 68 to account for quotes
+ echo "\"$subject\""
+ else
+ echo "\"$subject\"" | fold -s -w 68 | sed '2,$s/^/ /'
+ fi
+}
+
+# Calculate duration if start time available
+calculate_duration() {
+ if [ -f "$CI_START_TIME" ]; then
+ local start_time=$(cat "$CI_START_TIME" 2>/dev/null || echo $(date +%s))
+ local end_time=$(date +%s)
+ local duration_sec=$((end_time - start_time))
+ local minutes=$((duration_sec / 60))
+ local seconds=$((duration_sec % 60))
+
+ if [ $minutes -gt 0 ]; then
+ echo "${minutes}m ${seconds}s"
+ else
+ echo "${seconds}s"
+ fi
+ else
+ echo "unknown"
+ fi
+}
+
+# Determine scope and header format with enhanced context-aware detection
+determine_scope() {
+ # Simple detection: if TESTS is set, it's kdevops validation
+ if [[ -n "${TESTS:-}" ]] || [[ -n "${LIMIT_TESTS:-}" ]] || [[ -n "${TESTS_PARAM:-}" ]]; then
+ echo "kdevops"
+ else
+ echo "tests"
+ fi
+}
+
+# Get kernel information
+get_kernel_info() {
+ # Try to get git describe from linux directory, fallback gracefully
+ local kernel_describe="unknown"
+ local kernel_hash="unknown"
+ local kernel_subject="unknown commit"
+
+ if [ -d "linux/.git" ]; then
+ kernel_describe=$(cd linux && git describe --tags --always --dirty 2>/dev/null || echo "unknown")
+ kernel_hash=$(cd linux && git rev-parse --short=12 HEAD 2>/dev/null || echo "unknown")
+ kernel_subject=$(cd linux && git log -1 --pretty=format:"%s" 2>/dev/null || echo "unknown commit")
+ fi
+
+ echo "$kernel_describe|$kernel_hash|$kernel_subject"
+}
+
+# Get kdevops information (from kdevops directory context)
+get_kdevops_info() {
+ # This will be run from the kdevops directory
+ local kdevops_hash=$(git rev-parse --short=12 HEAD 2>/dev/null || echo "unknown")
+ local kdevops_subject=$(git log -1 --pretty=format:"%s" 2>/dev/null || echo "unknown commit")
+
+ echo "$kdevops_hash|$kdevops_subject"
+}
+
+# Read test results
+get_test_results() {
+ local result_content=""
+ local test_result="unknown"
+
+ if [ -f "$CI_COMMIT_EXTRA" ]; then
+ result_content=$(cat "$CI_COMMIT_EXTRA")
+ else
+ result_content="No test results available."
+ fi
+
+ if [ -f "$CI_RESULT" ]; then
+ test_result=$(cat "$CI_RESULT" | tr -d '\n' | tr -d '\r')
+ fi
+
+ echo "$test_result|$result_content"
+}
+
+# Determine workflow type for result formatting
+get_workflow_type() {
+ case "$CI_WORKFLOW" in
+ *xfs*|*btrfs*|*ext4*|*tmpfs*|*lbs-xfs*) echo "fstests" ;;
+ blktests*) echo "blktests" ;;
+ selftests*|*firmware*|*modules*|*mm*) echo "selftests" ;;
+ ai_*) echo "ai" ;;
+ mmtests_*) echo "mmtests" ;;
+ ltp_*) echo "ltp" ;;
+ fio-tests*|fio_*) echo "fio-tests" ;;
+ minio_*) echo "minio" ;;
+ *) echo "other" ;;
+ esac
+}
+
+# Generate the enhanced commit message
+generate_commit_message() {
+ local scope=$(determine_scope)
+ local workflow_type=$(get_workflow_type)
+ local duration=$(calculate_duration)
+
+ # Get kernel tree and ref from metadata files
+ local actual_kernel_tree="$KERNEL_TREE"
+ local actual_kernel_ref="unknown"
+
+ if [ -f "$CI_TRIGGER" ]; then
+ actual_kernel_tree=$(cat "$CI_TRIGGER" | tr -d '\n' | tr -d '\r')
+ fi
+
+ if [ -f "$CI_REF" ]; then
+ actual_kernel_ref=$(cat "$CI_REF" | tr -d '\n' | tr -d '\r')
+ fi
+
+ # Get kernel info (this assumes we're in the kernel directory context)
+ local kernel_info=$(get_kernel_info)
+ local kernel_describe=$(echo "$kernel_info" | cut -d'|' -f1)
+ local kernel_hash=$(echo "$kernel_info" | cut -d'|' -f2)
+ local kernel_subject=$(echo "$kernel_info" | cut -d'|' -f3)
+
+ # Use metadata ref if available, fallback to git describe
+ # Ensure we use the most accurate kernel version consistently
+ if [ "$actual_kernel_ref" != "unknown" ] && [ "$actual_kernel_ref" != "" ]; then
+ kernel_describe="$actual_kernel_ref"
+ fi
+
+ # Debug kernel version information
+
+ # Get kdevops info (this assumes we can access kdevops directory)
+ local kdevops_info=$(get_kdevops_info)
+ local kdevops_hash=$(echo "$kdevops_info" | cut -d'|' -f1)
+ local kdevops_subject=$(echo "$kdevops_info" | cut -d'|' -f2)
+
+ # Get test results
+ local results_info=$(get_test_results)
+ local test_result=$(echo "$results_info" | cut -d'|' -f1)
+ local result_content=$(echo "$results_info" | cut -d'|' -f2-)
+
+ # Build header
+ local header=""
+ if [ "$scope" = "kdevops" ]; then
+ # kdevops-ci validation format: focus on kdevops commit being tested
+ header="kdevops-ci: $CI_WORKFLOW: $kdevops_hash $kdevops_subject"
+ else
+ # Full test suite format: include PASS/FAIL status
+ local status="PASS"
+ if [ "$test_result" = "not ok" ] || [ "$test_result" = "fail" ]; then
+ status="FAIL"
+ fi
+ header="$CI_WORKFLOW ($actual_kernel_tree $kernel_describe): $status"
+ fi
+
+ # Build scope description
+ local scope_desc=""
+ if [ "$scope" = "kdevops" ]; then
+ local test_param="${TESTS:-${LIMIT_TESTS:-${TESTS_PARAM:-}}}"
+ if [[ -n "$test_param" ]]; then
+ scope_desc="kdevops validation (single test: $test_param)"
+ else
+ scope_desc="kdevops validation"
+ fi
+ else
+ scope_desc="full test suite"
+ fi
+
+ # Wrap kernel commit subject
+ local wrapped_kernel_subject=$(wrap_commit_subject "$kernel_subject")
+ local wrapped_kdevops_subject=$(wrap_commit_subject "$kdevops_subject")
+
+ # Generate metadata line with 72-char handling
+ local metadata_line="workflow: $CI_WORKFLOW | tree: $actual_kernel_tree | ref: $kernel_describe | scope: $scope | result: $test_result"
+ local metadata_output=""
+ if [ ${#metadata_line} -gt 72 ]; then
+ metadata_output="workflow: $CI_WORKFLOW | tree: $actual_kernel_tree"$'\n'"ref: $kernel_describe | scope: $scope | result: $test_result"
+ else
+ metadata_output="$metadata_line"
+ fi
+
+ # Build kernel validation info
+ local kernel_info_line=""
+ if [ "$actual_kernel_ref" != "unknown" ] && [ "$kernel_describe" != "$actual_kernel_ref" ]; then
+ # Show both requested and actual if they differ
+ kernel_info_line=" Kernel: $actual_kernel_ref (requested) → $kernel_describe (actual)"
+ else
+ # Show just the kernel version if they match or no metadata
+ kernel_info_line=" Kernel: $kernel_describe"
+ fi
+
+ # Build the complete commit message based on scope
+ if [ "$scope" = "kdevops" ]; then
+ # kdevops-ci validation format
+ cat << EOF
+$header
+
+BUILD INFO:
+ kdevops: $kdevops_hash ($wrapped_kdevops_subject)
+ Workflow: $CI_WORKFLOW
+ Test: ${TESTS_PARAM:-"complete suite"}
+$kernel_info_line
+ Duration: $duration
+
+EXECUTION RESULTS:
+$result_content
+
+METADATA:
+workflow: $CI_WORKFLOW | scope: kdevops | test: ${TESTS_PARAM:-"complete suite"} | requested: $actual_kernel_ref | actual: $kernel_describe | result: $test_result
+EOF
+ else
+ # Full test suite format
+ cat << EOF
+$header
+
+BUILD INFO:
+$kernel_info_line ($wrapped_kernel_subject)
+ Workflow: $CI_WORKFLOW
+ kdevops: $kdevops_hash ($wrapped_kdevops_subject)
+ Scope: $scope_desc
+ Duration: $duration
+
+EXECUTION RESULTS:
+$result_content
+
+METADATA:
+$metadata_output | requested: $actual_kernel_ref | actual: $kernel_describe
+EOF
+ fi
+}
+
+# Main execution
+main() {
+ # Validate required environment
+ if [ -z "$CI_WORKFLOW" ]; then
+ echo "Error: CI_WORKFLOW environment variable is required" >&2
+ exit 1
+ fi
+
+ generate_commit_message
+}
+
+# Run main function
+main "$@"
\ No newline at end of file
diff --git a/scripts/github_output.sh b/scripts/github_output.sh
new file mode 100755
index 00000000..711306c3
--- /dev/null
+++ b/scripts/github_output.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+#
+# Usage: ./github_output.sh key value
+set -euxo pipefail
+
+key="$1"
+value="$2"
+
+echo "$key=$value" >> "$GITHUB_OUTPUT"
diff --git a/scripts/korg-releases.py b/scripts/korg-releases.py
new file mode 100755
index 00000000..de494327
--- /dev/null
+++ b/scripts/korg-releases.py
@@ -0,0 +1,94 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: copyleft-next-0.3.1
+import sys
+import logging
+import argparse
+import json
+import urllib.request
+import socket
+import re
+
+
+def parser():
+ parser = argparse.ArgumentParser(description="kernel.org/releases.json checker")
+ parser.add_argument("--debug", action="store_true", help="debug")
+ parser.add_argument(
+ "--moniker",
+ help="moniker (mainline, stable, longterm or linux-next)",
+ required=True,
+ )
+ parser.add_argument(
+ "--pname",
+ help="project name for User-Agent request",
+ default="kdevops",
+ )
+ parser.add_argument(
+ "--pversion",
+ help="project version for User-Agent request",
+ default="5.0.2",
+ )
+ return parser
+
+
+def _check_connection(host, port, timeout=2):
+ try:
+ with socket.create_connection((host, port), timeout):
+ logging.debug(f"Connection to {host} on port {port} succeeded!")
+ return True
+ except (socket.timeout, socket.error) as e:
+ logging.debug(f"Connection to {host} on port {port} failed: {e}")
+ return False
+
+
+def kreleases(args) -> None:
+ """Get the latest kernel releases from kernel.org/releases.json"""
+
+ reflist = []
+ if _check_connection("kernel.org", 80):
+ _url = "https://www.kernel.org/releases.json"
+ req = urllib.request.Request(
+ _url,
+ headers={
+ "User-Agent": f"{args.pname}/{args.pversion} (kdevops@lists.linux.dev)"
+ },
+ )
+ with urllib.request.urlopen(req) as url:
+ data = json.load(url)
+
+ for release in data["releases"]:
+ if release["moniker"] == args.moniker:
+ # Check if release.json is aa.bb.cc type
+ if re.compile(r"^\d+\.\d+(\.\d+|-rc\d+)?$").match(
+ release["version"]
+ ):
+ reflist.append("v" + release["version"])
+ else:
+ reflist.append(release["version"])
+
+ logging.debug(f"{reflist}")
+ for r in reflist:
+ print(r)
+
+
+def main() -> None:
+ """Kconfig choice generator for git refereces"""
+ log = logging.getLogger()
+ log.setLevel(logging.INFO)
+ p = parser()
+ args, _ = p.parse_known_args()
+ if args.debug:
+ log.setLevel(logging.DEBUG)
+
+ kreleases(args)
+
+
+if __name__ == "__main__":
+ ret = 0
+ try:
+ main()
+ except Exception:
+ ret = 1
+ import traceback
+
+ traceback.print_exc()
+ sys.exit(ret)
--
2.50.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH RFC 0/8] Refactor CI
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
` (7 preceding siblings ...)
2025-09-23 21:13 ` [PATCH RFC 8/8] github: refactor into reusable actions and workflows Daniel Gomez
@ 2025-09-24 1:14 ` Luis Chamberlain
8 siblings, 0 replies; 10+ messages in thread
From: Luis Chamberlain @ 2025-09-24 1:14 UTC (permalink / raw)
To: Daniel Gomez; +Cc: Chuck Lever, kdevops, Daniel Gomez, dagomez
On Tue, Sep 23, 2025 at 11:13:04PM +0200, Daniel Gomez wrote:
> Refactor the .github/ CI setup into modular and reusable workflows and
> actions. This makes it possible to share modules from kdevops repo and
> reuse them in other GitHub projects.
Thanks! Applied and pushed.
Luis
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2025-09-24 1:14 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 1/8] base_image: optimize VM image copying with reflinks and fix same-file handling Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 2/8] kconfig: add KDEVOPS_MAKE_VERBOSE option for persistent verbose output Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 3/8] defconfig: add ci fragment Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 4/8] ansible: enhance DIY callback output format for improved readability Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 5/8] ansible: add profile_tasks callback timing analysis support Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 6/8] scripts: fix CI_WORKFLOW basename mapping Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 7/8] fstests: fix ansible_host undefined error Daniel Gomez
2025-09-23 21:13 ` [PATCH RFC 8/8] github: refactor into reusable actions and workflows Daniel Gomez
2025-09-24 1:14 ` [PATCH RFC 0/8] Refactor CI Luis Chamberlain
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox