* [PATCH RFC 8/8] github: refactor into reusable actions and workflows
2025-09-23 21:13 [PATCH RFC 0/8] Refactor CI Daniel Gomez
` (6 preceding siblings ...)
2025-09-23 21:13 ` [PATCH RFC 7/8] fstests: fix ansible_host undefined error Daniel Gomez
@ 2025-09-23 21:13 ` Daniel Gomez
2025-09-24 1:14 ` [PATCH RFC 0/8] Refactor CI Luis Chamberlain
8 siblings, 0 replies; 10+ messages in thread
From: Daniel Gomez @ 2025-09-23 21:13 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez
From: Daniel Gomez <da.gomez@samsung.com>
Refactor the .github/ CI support into reusable actions and workflows.
This allows to use the new actions not only internally in kdevops but
also in other projects such linux-modules-kpd and similar. Keeping the
ci logic in one place and allowing other projects to get the latest
changes.
CI supports the following:
- Split workflow in setup, test, archive and cleanup actions
- Fixed kernel: v6.15 on linux tree. This is to speed up builds relying
on the same kernel. We can update when needed or timely basis
- Fixed workflows: blktests_nvme and xfs_reflink_4k. We want to validate
kdevops workflows and blktests with nvme section/profile has been added
on top of the current fstests/xfs reflink 4k
- 2 CI modes:
A. kdevops selftest/ci validation. A single test execution for the given
workflow (e.g. generic/003 for xfs). This has it's own commit format.
B. kdevops full workflow. This is to validate a specific kernel on
a given test suite. It's supported on manual trigger as well as on
schedule.
Add automatic runner assignment based on ci_workflow names to enable
parallel execution on dedicated runners:
- Parallelize GitHub workflow execution on dedicated runners based on
CI_WORKFLOW names (using tags).
E.g.:
* blktests workflows → [self-hosted, Linux, X64, blktests]
* xfs workflows → [self-hosted, Linux, X64, xfs]
- Set timeout 120 minutes for CI B mode.
- Disable fail-fast to matrix strategy in both push.yml and schedule.yml
workflows to prevent cancellation of successful jobs when other matrix
jobs fail. Example: this allows blktests_nvme to continue running even
if xfs_reflink_4k fails
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
.github/actions/archive/action.yml | 28 ++++
.github/actions/cleanup/action.yml | 24 ++++
.github/actions/setup/action.yml | 187 ++++++++++++++++++++++++
.github/actions/test/action.yml | 213 +++++++++++++++++++++++++++
.github/workflows/destroy.yml | 93 ++++++++++++
.github/workflows/docker-tests.yml | 2 +-
.github/workflows/fstests.yml | 98 -------------
.github/workflows/main.yml | 97 +++++++++++++
.github/workflows/manual.yml | 122 ++++++++++++++++
.github/workflows/push.yml | 29 ++++
.github/workflows/schedule.yml | 54 +++++++
scripts/generate_ci_commit_message.sh | 263 ++++++++++++++++++++++++++++++++++
scripts/github_output.sh | 9 ++
scripts/korg-releases.py | 94 ++++++++++++
14 files changed, 1214 insertions(+), 99 deletions(-)
diff --git a/.github/actions/archive/action.yml b/.github/actions/archive/action.yml
new file mode 100644
index 00000000..6efe20fc
--- /dev/null
+++ b/.github/actions/archive/action.yml
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Archive results
+description: Archive kdevops results in https://github.com/linux-kdevops/kdevops-results-archive.git
+inputs:
+ ci_workflow:
+ required: false
+ type: string
+ default: 'demo'
+ dir:
+ description: 'Directory'
+ required: true
+ default: 'workdir'
+
+runs:
+ using: "composite"
+ steps:
+ - name: Get systemd journal files
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ make journal-dump
+
+ - name: Build our kdevops archive results
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ make ci-archive CI_WORKFLOW="${{ inputs.ci_workflow }}"
diff --git a/.github/actions/cleanup/action.yml b/.github/actions/cleanup/action.yml
new file mode 100644
index 00000000..18a8656a
--- /dev/null
+++ b/.github/actions/cleanup/action.yml
@@ -0,0 +1,24 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Cleanup kdevops VMs
+description: Destroy VMs and cleanup workspace
+
+inputs:
+ dir:
+ description: 'Directory'
+ required: true
+ default: 'workdir'
+
+runs:
+ using: "composite"
+ steps:
+ - name: Run kdevops make destroy
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ make destroy
+
+ - name: Remove working-directory
+ shell: bash
+ run: |
+ rm --recursive --force --verbose ${{ inputs.dir }}
diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml
new file mode 100644
index 00000000..407c8ac3
--- /dev/null
+++ b/.github/actions/setup/action.yml
@@ -0,0 +1,187 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Setup kdevops
+description: Setup kdevops workspace
+
+inputs:
+ ci_workflow:
+ required: false
+ type: string
+ default: 'demo'
+ dir:
+ description: 'Directory'
+ required: true
+ default: 'workdir'
+ kernel_tree:
+ required: false
+ type: string
+ default: 'linux'
+ kernel_ref:
+ required: false
+ type: string
+ default: 'master'
+
+runs:
+ using: "composite"
+ steps:
+ - name: Create workspace directory
+ shell: bash
+ run: |
+ set -euxo pipefail
+ rm --recursive --force --verbose ${{ inputs.dir }}
+ mkdir --parent --verbose ${{ inputs.dir }}
+
+ - name: Configure git
+ shell: bash
+ run: |
+ set -euxo pipefail
+ git config --global --add safe.directory '*'
+ git config --global user.name "kdevops"
+ git config --global user.email "kdevops@lists.linux.dev"
+
+ - name: Checkout kdevops
+ working-directory: ${{ inputs.dir }}
+ shell: bash
+ run: |
+ set -euxo pipefail
+ git clone https://github.com/dkruces/kdevops.git --branch main kdevops
+
+ - name: Checkout custom branch with delta on kdevops/linux
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ LINUX_TREE="/mirror/${{ inputs.kernel_tree }}.git"
+ LINUX_TREE_REF="${{ inputs.kernel_ref }}"
+ git clone $LINUX_TREE linux
+ cd linux
+ git checkout $LINUX_TREE_REF
+ git log -1
+
+ - name: Make sure our repo kdevops defconfig exists
+ id: defconfig
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ KDEVOPS_DEFCONFIG=${{ inputs.ci_workflow }}
+
+ if [[ ! -f defconfigs/$KDEVOPS_DEFCONFIG ]]; then
+ echo "Missing defconfig: defconfigs/$KDEVOPS_DEFCONFIG"
+ exit 1
+ fi
+
+ "${{ github.workspace }}/scripts/github_output.sh" \
+ KDEVOPS_DEFCONFIG "$KDEVOPS_DEFCONFIG"
+
+ - name: Initialize CI metadata for kdevops-results-archive for linux
+ id: metadata
+ working-directory: ${{ inputs.dir }}/kdevops/linux
+ shell: bash
+ run: |
+ set -euxo pipefail
+ echo "${{ inputs.kernel_tree }}" > ../ci.trigger
+ # Get the kdevops commit subject, not kernel commit
+ cd "${{ github.workspace }}" && git log -1 --pretty=format:"%s" > "${{ inputs.dir }}/kdevops/ci.subject"
+ cd - > /dev/null
+ echo "${{ inputs.kernel_ref }}" > ../ci.ref
+
+ RELEVANT_GIT_TAG=$(cat ../ci.ref)
+ RELEVANT_GIT_REF=$(git rev-parse --short=12 $RELEVANT_GIT_TAG)
+
+ "${{ github.workspace }}/scripts/github_output.sh" \
+ LINUX_GIT_REF "$RELEVANT_GIT_REF"
+ "${{ github.workspace }}/scripts/github_output.sh" \
+ LINUX_GIT_TAG "$RELEVANT_GIT_TAG"
+
+ # Start out pessimistic
+ echo "unknown" > ../ci.result
+ echo "Nothing to write home about." > ../ci.commit_extra
+
+ - name: Run kdevops make defconfig-repo
+ working-directory: ${{ inputs.dir }}/kdevops
+ env:
+ LINUX_GIT_TAG: ${{ steps.metadata.outputs.LINUX_GIT_TAG }}
+ LINUX_GIT_REF: ${{ steps.metadata.outputs.LINUX_GIT_REF }}
+ KDEVOPS_DEFCONFIG: ${{ steps.defconfig.outputs.KDEVOPS_DEFCONFIG }}
+ shell: bash
+ run: |
+ set -euxo pipefail
+ LINUX_TREE="/mirror/${{ inputs.kernel_tree }}.git"
+ LINUX_TREE_REF="$LINUX_GIT_TAG"
+
+ # We make the compromise here to use a relevant git tag for the
+ # host prefix so that folks can easily tell what exact kernel tree
+ # is being tested by using the relevant git ref. That is, if you
+ # pushed a tree with the .github/ directory as the top of the tree,
+ # that commit will not be used, we'll use the last one as that is
+ # the relevant git ref we want to annotate a test for.
+ #
+ # The compromise here is that we expect no two same identical tests
+ # on the same self-hosted server. We could extend this with something
+ # like github.run_id but its not yet clear if we can have kdevops
+ # hosts with a bundled prefix ID like that ref-runid-testname. Tests
+ # have been done with the full lenght sha1sum though and we know that
+ # does work.
+ KDEVOPS_HOSTS_PREFIX="kci-$LINUX_GIT_REF"
+
+ echo "Going to use defconfig-$KDEVOPS_DEFCONFIG"
+
+ echo "Linux tree: $LINUX_TREE"
+ echo "Linux trigger ref: $LINUX_TREE_REF"
+ echo "Linux tag: $LINUX_GIT_TAG"
+ echo "Runner ID: ${{ github.run_id }}"
+ echo "kdevops host prefix: $KDEVOPS_HOSTS_PREFIX"
+ echo "kdevops defconfig: defconfig-$KDEVOPS_DEFCONFIG"
+
+ # Customize KMOD_TIMEOUT when required
+ KMOD_TIMEOUT_ARG=
+ if [[ "$(hostname)" == *smc111* && \
+ "$KDEVOPS_DEFCONFIG" == "linux-modules-kpd" ]]; then
+ KMOD_TIMEOUT_ARG="KMOD_TIMEOUT=222"
+ fi
+
+ KDEVOPS_ARGS="\
+ KDEVOPS_HOSTS_PREFIX=$KDEVOPS_HOSTS_PREFIX \
+ LINUX_TREE=$LINUX_TREE \
+ LINUX_TREE_REF=$LINUX_TREE_REF \
+ ${KMOD_TIMEOUT_ARG} \
+ defconfig-$KDEVOPS_DEFCONFIG"
+ echo "Going to run:"
+ echo "make $KDEVOPS_ARGS"
+
+ make $KDEVOPS_ARGS
+ ./scripts/kconfig/merge_config.sh \
+ -n .config \
+ defconfigs/configs/diy.config \
+ defconfigs/configs/ci.config
+
+ - name: Run kdevops make
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ make -j$(nproc)
+
+ - name: Run kdevops make bringup
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ ls -ld linux
+ make destroy
+ make bringup
+
+ - name: Build linux and boot test nodes on test kernel
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ make linux
+
+ - name: Build required ci tests
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+ make ci-build-test CI_WORKFLOW=${{ inputs.ci_workflow }}
diff --git a/.github/actions/test/action.yml b/.github/actions/test/action.yml
new file mode 100644
index 00000000..a9022b4c
--- /dev/null
+++ b/.github/actions/test/action.yml
@@ -0,0 +1,213 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Setup kdevops
+description: Setup kdevops workspace
+
+inputs:
+ dir:
+ description: 'Directory'
+ required: true
+ default: 'workdir'
+ ci_workflow:
+ required: false
+ type: string
+ default: 'demo'
+ test_mode:
+ description: 'Testing mode'
+ required: false
+ default: 'full-testing'
+ tests:
+ description: 'Custom test to run (for kdevops-validation mode only)'
+ required: false
+ default: ''
+
+runs:
+ using: "composite"
+ steps:
+ - name: Run CI tests
+ id: ci_test
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+
+ # Create start time for duration calculation
+ echo "$(date +%s)" > ci.start_time
+
+ # Handle test mode and tests parameters
+ if [[ -n "${{ inputs.tests }}" ]]; then
+ # Custom test specified for kdevops-validation mode
+ TESTS="${{ inputs.tests }}"
+ elif [[ "${{ inputs.test_mode }}" == "kdevops-validation" ]]; then
+ # Auto-assign appropriate test based on workflow
+ case "${{ inputs.ci_workflow }}" in
+ *fstests*|*xfs*|*btrfs*|*ext4*|*tmpfs*)
+ TESTS="generic/003"
+ ;;
+ blktests*)
+ TESTS="block/003"
+ ;;
+ selftests*|*modules*|*kmod*|*firmware*|*mm*)
+ TESTS="kmod/test_001"
+ ;;
+ *)
+ TESTS="generic/003"
+ ;;
+ esac
+ else
+ # Full testing mode - no TESTS variable
+ TESTS=""
+ fi
+
+ # Export TESTS for current step and save to GitHub output for later steps
+ export TESTS="$TESTS"
+ "${{ github.workspace }}/scripts/github_output.sh" TESTS "$TESTS"
+
+ make ci-test CI_WORKFLOW="${{ inputs.ci_workflow }}"
+
+ - name: Generate workflow results path
+ id: setpath
+ shell: bash
+ run: |
+ set -euxo pipefail
+
+ case "${{ inputs.ci_workflow }}" in
+ blktests*) wpath="workflows/blktests" ;;
+ *btrfs*) wpath="workflows/fstests" ;;
+ *ext4*) wpath="workflows/fstests" ;;
+ tmpfs*) wpath="workflows/fstests" ;;
+ *xfs*) wpath="workflows/fstests" ;;
+ *) wpath="workflows/selftests" ;;
+ esac
+
+ "${{ github.workspace }}/scripts/github_output.sh" wpath "$wpath"
+
+ - name: Generate CI commit info with workflow-specific logic
+ working-directory: ${{ inputs.dir }}/kdevops
+ shell: bash
+ run: |
+ set -euxo pipefail
+
+ wpath="${{ steps.setpath.outputs.wpath }}"
+
+ # Workflow-specific result collection
+ case "${{ inputs.ci_workflow }}" in
+ *xfs*|*btrfs*|*ext4*|tmpfs*|*fstests*)
+ # fstests workflows: Use xunit_results.txt for rich summary
+ echo "Collecting fstests results..."
+ if find "$wpath/results/last-run" -name "xunit_results.txt" -type f -exec cat {} \; > ci.commit_extra 2>/dev/null; then
+ echo "Found xunit_results.txt"
+ else
+ echo "No xunit_results.txt found, using fallback..."
+ echo "Kernel tests results:" > ci.commit_extra
+ find "$wpath/results/last-run/" -name '*.dmesg.log' -exec tail -n 1 {} + >> ci.commit_extra 2>/dev/null || true
+ fi
+
+ # fstests success detection
+ if ! grep -E "failures, [1-9]|errors, [1-9]" ci.commit_extra >/dev/null 2>&1; then
+ echo "ok" > ci.result
+ else
+ echo "not ok" > ci.result
+ fi
+ ;;
+ blktests*)
+ # blktests workflows: Collect individual test results
+
+ echo "Kernel tests results:" > ci.commit_extra
+
+ # Collect test results from last-run directory
+ if [ -d "$wpath/results/last-run" ]; then
+ echo -e "\nBlktests summary:" >> ci.commit_extra
+
+ # Count total tests by unique test names (not file extensions)
+ # Get base test names by removing extensions and extracting just the test name
+ test_names=$(find "$wpath/results/last-run" -type f -name "*" | \
+ grep -E '/[^/]*$' | \
+ sed 's|.*/||' | \
+ sed 's/\.\(full\|out\|runtime\|dmesg\)$//' | \
+ sort -u)
+ total_tests=$(echo "$test_names" | grep -v '^$' | wc -l)
+
+ bad_files=$(find "$wpath/results/last-run" -name "*.out.bad" -type f)
+ if [[ -n "$bad_files" ]]; then
+ failed_tests=$(echo "$bad_files" | wc -l)
+ else
+ failed_tests=0
+ fi
+
+ passed_tests=$((total_tests - failed_tests))
+
+ echo "Tests run: $total_tests, Passed: $passed_tests, Failed: $failed_tests" >> ci.commit_extra
+
+ # List failed tests if any
+ if [ $failed_tests -gt 0 ]; then
+ echo -e "\nFailed tests:" >> ci.commit_extra
+ find "$wpath/results/last-run" -name "*.out.bad" -type f | sed 's|.*/\([^/]*\)\.out\.bad$|\1|' | sort >> ci.commit_extra 2>/dev/null || true
+ fi
+
+ # Show sample test status files for passed tests
+ if [ $passed_tests -gt 0 ]; then
+ echo -e "\nSample passed test:" >> ci.commit_extra
+ sample_files=$(find "$wpath/results/last-run" -type f -name "*" ! -name "*.out.bad" ! -name "*.dmesg")
+ sample_file=$(echo "$sample_files" | head -1)
+ if [ -n "$sample_file" ] && [ -f "$sample_file" ]; then
+ cat "$sample_file" >> ci.commit_extra || echo "Failed to read sample file" >> ci.commit_extra
+ else
+ echo "No valid sample file found" >> ci.commit_extra
+ fi
+ fi
+ else
+ echo -e "\nNo blktests results found in $wpath/results/last-run" >> ci.commit_extra
+ fi
+
+ # blktests success detection - look for .out.bad files (failures)
+ bad_check=$(find "$wpath/results/last-run" -name "*.out.bad" -type f | head -1)
+
+ if [ -n "$bad_check" ]; then
+ echo "not ok" > ci.result
+ else
+ echo "ok" > ci.result
+ fi
+ ;;
+ *selftests*|*modules*|*mm*|*firmware*)
+ # selftests workflows: Use userspace.log
+ echo "Kernel tests results:" > ci.commit_extra
+ find "$wpath/results/last-run/" -name '*.userspace.log' -exec cat {} \; >> ci.commit_extra 2>/dev/null || true
+
+ # selftests success detection
+ if grep -q "passed\|PASS" ci.commit_extra && ! grep -q "FAIL\|failed\|ERROR" ci.commit_extra; then
+ echo "ok" > ci.result
+ else
+ echo "not ok" > ci.result
+ fi
+ ;;
+ *)
+ # Default/unknown workflows: Generic approach
+ echo "Kernel tests results:" > ci.commit_extra
+ find "$wpath/results/last-run/" -name '*.dmesg.log' -exec tail -n 1 {} + >> ci.commit_extra 2>/dev/null || true
+ echo -e "\n\nUserspace test results:" >> ci.commit_extra
+ find "$wpath/results/last-run/" -name '*.userspace.log' -exec tail -n 1 {} + >> ci.commit_extra 2>/dev/null || true
+
+ if grep -i -q "fail" ci.commit_extra; then
+ echo "not ok" > ci.result
+ else
+ echo "ok" > ci.result
+ fi
+ ;;
+ esac
+
+ # Set environment variables for the commit message script
+ export CI_WORKFLOW="${{ inputs.ci_workflow }}"
+ export KERNEL_TREE="${{ inputs.kernel_tree || 'linux' }}"
+
+ # Get TESTS from GitHub output (determines kdevops validation vs full testing)
+ TESTS="${{ steps.ci_test.outputs.TESTS }}"
+ if [[ -n "${TESTS:-}" ]]; then
+ export TESTS="$TESTS"
+ fi
+
+ # Generate enhanced commit message using our script
+ ./scripts/generate_ci_commit_message.sh > ci.commit_message_enhanced
+
+ # Keep the original ci.commit_extra for backward compatibility
+ # The archive action will use ci.commit_message_enhanced if available
diff --git a/.github/workflows/destroy.yml b/.github/workflows/destroy.yml
new file mode 100644
index 00000000..6689452b
--- /dev/null
+++ b/.github/workflows/destroy.yml
@@ -0,0 +1,93 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Run kdevops Destroyer - Manual
+
+on:
+ workflow_dispatch:
+ inputs:
+ ci_workflow:
+ description: "CI Workflow"
+ required: true
+ default: 'blktests_nvme'
+ type: choice
+ options:
+ - blktests
+ - blktests_block
+ - blktests_loop
+ - blktests_meta
+ - blktests_nbd
+ - blktests_nvme
+ - blktests_nvmemp
+ - blktests_scsi
+ - blktests_srp
+ - blktests_zbd
+ - lbs-xfs
+ - lbs-xfs-bdev-large-nvme
+ - lbs-xfs-bdev-nvme
+ - lbs-xfs-small
+ - xfs
+ - xfs_crc
+ - xfs_crc_logdev
+ - xfs_crc_rtdev
+ - xfs_crc_rtdev_extsize_28k
+ - xfs_crc_rtdev_extsize_64k
+ - xfs_nocrc
+ - xfs_nocrc_16k_4ks
+ - xfs_nocrc_1k
+ - xfs_nocrc_2k
+ - xfs_nocrc_32_4ks
+ - xfs_nocrc_4k
+ - xfs_nocrc_512
+ - xfs_nocrc_64_4ks
+ - xfs_nocrc_8k_4ks
+ - xfs_nocrc_lbs
+ - xfs_reflink
+ - xfs_reflink_1024
+ - xfs_reflink_16k_4ks
+ - xfs_reflink_2k
+ - xfs_reflink_32k_4ks
+ - xfs_reflink_4k
+ - xfs_reflink_4k_8ks
+ - xfs_reflink_64k_4ks
+ - xfs_reflink_dir_bsize_8k
+ - xfs_reflink_lbs
+ - xfs_reflink_logdev
+ - xfs_reflink_normapbt
+ - xfs_reflink_nrext64
+ - xfs_reflink_stripe_len
+ - xfs-soak
+ host_prefix:
+ description: "Host prefix"
+ required: true
+ default: 'debian13'
+ type: string
+
+jobs:
+ destroyer:
+ name: Destroy guests
+ runs-on: [self-hosted]
+ steps:
+ - name: Create workdir
+ run: |
+ rm --recursive --force --verbose ${{ inputs.ci_workflow }}
+ mkdir --parent --verbose ${{ inputs.ci_workflow }}
+
+ - name: Checkout kdevops
+ run: |
+ cd ${{ inputs.ci_workflow }}
+ git clone https://github.com/dkruces/kdevops.git --branch ci-workflow kdevops
+
+ - name: Run kdevops make defconfig-repo
+ run: |
+ set -euxo pipefail
+
+ sudo virsh list --all
+
+ cd ${{ inputs.ci_workflow }}/kdevops
+ make \
+ KDEVOPS_HOSTS_PREFIX="${{ inputs.host_prefix }}" \
+ defconfig-${{ inputs.ci_workflow }}
+
+ make destroy
+
+ sudo virsh list --all
diff --git a/.github/workflows/docker-tests.yml b/.github/workflows/docker-tests.yml
index c0e0d03d..33d67f09 100644
--- a/.github/workflows/docker-tests.yml
+++ b/.github/workflows/docker-tests.yml
@@ -1,4 +1,4 @@
-name: Setup kdevops
+name: kdevops Docker Tests
on:
push:
diff --git a/.github/workflows/fstests.yml b/.github/workflows/fstests.yml
deleted file mode 100644
index e13978c7..00000000
--- a/.github/workflows/fstests.yml
+++ /dev/null
@@ -1,98 +0,0 @@
-name: Run kdevops on self-hosted runner
-
-on:
- push:
- branches:
- - '**'
- pull_request:
- branches:
- - '**'
- workflow_dispatch: # Add this for manual triggering of the workflow
-
-jobs:
- run-kdevops:
- name: Run kdevops CI
- runs-on: [self-hosted, Linux, X64]
- steps:
- - name: Checkout repository
- uses: actions/checkout@v4
-
- - name: Set CI metadata for kdevops-results-archive
- run: |
- echo "$(basename ${{ github.repository }})" > ci.trigger
- git log -1 --pretty=format:"%s" > ci.subject
- # Start out pessimistic
- echo "not ok" > ci.result
- echo "Nothing to write home about." > ci.commit_extra
-
- - name: Set kdevops path
- run: echo "KDEVOPS_PATH=$GITHUB_WORKSPACE" >> $GITHUB_ENV
-
- - name: Configure git
- run: |
- git config --global --add safe.directory '*'
- git config --global user.name "kdevops"
- git config --global user.email "kdevops@lists.linux.dev"
-
- - name: Run kdevops make defconfig-repo
- run: |
- KDEVOPS_TREE_REF="${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}"
- SHORT_PREFIX="$(echo ${KDEVOPS_TREE_REF:0:12})"
- make KDEVOPS_HOSTS_PREFIX="$SHORT_PREFIX" \
- ANSIBLE_CFG_CALLBACK_PLUGIN="debug" \
- LINUX_TREE_REF="v6.15" \
- defconfig-xfs_reflink_4k
-
- - name: Run kdevops make
- run: |
- make V=1 -j$(nproc)
-
- - name: Run kdevops make bringup
- run: |
- make V=1 bringup
-
- - name: Build linux and boot test nodes on test kernel
- run: |
- make V=1 linux
-
- - name: Build fstests
- run: |
- make V=1 fstests
-
- - name: Run just one fstest to verify we tests and test collection works
- run: |
- make V=1 fstests-baseline TESTS=generic/003
- echo "ok" > ci.result
- find workflows/fstests/results/last-run -name xunit_results.txt -type f -exec cat {} \; > ci.commit_extra || true
- if ! grep -E "failures, [1-9]|errors, [1-9]" ci.commit_extra; then
- echo "ok" > ci.result
- fi
-
- - name: Get systemd journal files
- if: always() # This ensures the step runs even if previous steps failed
- run: |
- make V=1 journal-dump
-
- - name: Start SSH Agent
- if: always() # Ensure this step runs even if previous steps failed
- uses: webfactory/ssh-agent@v0.9.0
- with:
- ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
-
- - name: Build our kdevops archive results
- if: always() # This ensures the step runs even if previous steps failed
- run: |
- make V=1 ci-archive
-
- - name: Upload our kdevops results archive
- if: always() # This ensures the step runs even if previous steps failed
- uses: actions/upload-artifact@v4
- with:
- name: kdevops-ci-results
- path: ${{ env.KDEVOPS_PATH }}/archive/*.zip
-
- # Ensure make destroy always runs, even on failure
- - name: Run kdevops make destroy
- if: always() # This ensures the step runs even if previous steps failed
- run: |
- make V=1 destroy
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
new file mode 100644
index 00000000..35423403
--- /dev/null
+++ b/.github/workflows/main.yml
@@ -0,0 +1,97 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Run kdevops CI Workflow - Reusable
+
+on:
+ workflow_call:
+ inputs:
+ ci_workflow:
+ description: "CI Workflow"
+ required: true
+ default: 'blktests_nvme'
+ type: string
+ kernel_ref:
+ description: "Linux tree git reference (branch/tag/commit-id)"
+ required: true
+ default: 'master'
+ type: string
+ kernel_tree:
+ description: "Linux kernel tree to use"
+ required: true
+ default: 'linux'
+ type: string
+ test_mode:
+ description: 'Testing mode'
+ required: false
+ default: 'full-testing'
+ type: string
+ tests:
+ description: 'Custom test to run (for kdevops-validation mode only)'
+ required: false
+ default: ''
+ type: string
+
+jobs:
+ setup:
+ name: Setup kdevops workspace
+ runs-on: ${{ contains(inputs.ci_workflow, 'blktests') && fromJSON('["self-hosted", "Linux", "X64", "blktests"]') || contains(inputs.ci_workflow, 'tmpfs') && fromJSON('["self-hosted", "Linux", "X64", "tmpfs"]') || contains(inputs.ci_workflow, 'xfs') && fromJSON('["self-hosted", "Linux", "X64", "xfs"]') || contains(inputs.ci_workflow, 'btrfs') && fromJSON('["self-hosted", "Linux", "X64", "btrfs"]') || contains(inputs.ci_workflow, 'ext4') && fromJSON('["self-hosted", "Linux", "X64", "ext4"]') || contains(inputs.ci_workflow, 'selftests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'seltests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'modules') && fromJSON('["self-hosted", "Linux", "X64", "modules"]') || contains(inputs.ci_workflow, 'firmware') && fromJSON('["self-hosted", "Linux", "X64", "firmware"]') || contains(inputs.ci_workflow, 'mm') && fromJSON('["self-hosted", "Linux", "X64", "mm"]') || fromJSON('["self-hosted"]') }}
+ steps:
+ - name: Checkout dkruces/kdevops
+ uses: actions/checkout@v4
+ with:
+ clean: false
+
+ - name: kdevops setup
+ uses: ./.github/actions/setup
+ with:
+ ci_workflow: ${{ inputs.ci_workflow }}
+ dir: ${{ inputs.ci_workflow }}
+ kernel_ref: ${{ inputs.kernel_ref }}
+ kernel_tree: ${{ inputs.kernel_tree }}
+
+ test:
+ name: Run kdevops ci-test
+ runs-on: ${{ contains(inputs.ci_workflow, 'blktests') && fromJSON('["self-hosted", "Linux", "X64", "blktests"]') || contains(inputs.ci_workflow, 'tmpfs') && fromJSON('["self-hosted", "Linux", "X64", "tmpfs"]') || contains(inputs.ci_workflow, 'xfs') && fromJSON('["self-hosted", "Linux", "X64", "xfs"]') || contains(inputs.ci_workflow, 'btrfs') && fromJSON('["self-hosted", "Linux", "X64", "btrfs"]') || contains(inputs.ci_workflow, 'ext4') && fromJSON('["self-hosted", "Linux", "X64", "ext4"]') || contains(inputs.ci_workflow, 'selftests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'seltests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'modules') && fromJSON('["self-hosted", "Linux", "X64", "modules"]') || contains(inputs.ci_workflow, 'firmware') && fromJSON('["self-hosted", "Linux", "X64", "firmware"]') || contains(inputs.ci_workflow, 'mm') && fromJSON('["self-hosted", "Linux", "X64", "mm"]') || fromJSON('["self-hosted"]') }}
+ needs: [setup]
+ timeout-minutes: 120
+ steps:
+ - name: kdevops ci-test
+ uses: ./.github/actions/test
+ with:
+ ci_workflow: ${{ inputs.ci_workflow }}
+ dir: ${{ inputs.ci_workflow }}
+ test_mode: ${{ inputs.test_mode }}
+ tests: ${{ inputs.tests }}
+
+ archive:
+ name: Archive kdevops
+ runs-on: ${{ contains(inputs.ci_workflow, 'blktests') && fromJSON('["self-hosted", "Linux", "X64", "blktests"]') || contains(inputs.ci_workflow, 'tmpfs') && fromJSON('["self-hosted", "Linux", "X64", "tmpfs"]') || contains(inputs.ci_workflow, 'xfs') && fromJSON('["self-hosted", "Linux", "X64", "xfs"]') || contains(inputs.ci_workflow, 'btrfs') && fromJSON('["self-hosted", "Linux", "X64", "btrfs"]') || contains(inputs.ci_workflow, 'ext4') && fromJSON('["self-hosted", "Linux", "X64", "ext4"]') || contains(inputs.ci_workflow, 'selftests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'seltests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'modules') && fromJSON('["self-hosted", "Linux", "X64", "modules"]') || contains(inputs.ci_workflow, 'firmware') && fromJSON('["self-hosted", "Linux", "X64", "firmware"]') || contains(inputs.ci_workflow, 'mm') && fromJSON('["self-hosted", "Linux", "X64", "mm"]') || fromJSON('["self-hosted"]') }}
+ needs: [setup, test]
+ steps:
+ - name: Start SSH Agent
+ uses: webfactory/ssh-agent@v0.9.0
+ with:
+ ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
+
+ - name: Archive ci-test results
+ uses: ./.github/actions/archive
+ with:
+ ci_workflow: ${{ inputs.ci_workflow }}
+ dir: ${{ inputs.ci_workflow }}
+
+ - name: Upload our kdevops results archive
+ uses: actions/upload-artifact@v4
+ with:
+ name: kdevops-ci-results
+ path: ${{ inputs.ci_workflow }}/kdevops/archive/*.zip
+
+ cleanup:
+ name: Cleanup kdevops workspace
+ runs-on: ${{ contains(inputs.ci_workflow, 'blktests') && fromJSON('["self-hosted", "Linux", "X64", "blktests"]') || contains(inputs.ci_workflow, 'tmpfs') && fromJSON('["self-hosted", "Linux", "X64", "tmpfs"]') || contains(inputs.ci_workflow, 'xfs') && fromJSON('["self-hosted", "Linux", "X64", "xfs"]') || contains(inputs.ci_workflow, 'btrfs') && fromJSON('["self-hosted", "Linux", "X64", "btrfs"]') || contains(inputs.ci_workflow, 'ext4') && fromJSON('["self-hosted", "Linux", "X64", "ext4"]') || contains(inputs.ci_workflow, 'selftests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'seltests') && fromJSON('["self-hosted", "Linux", "X64", "selftests"]') || contains(inputs.ci_workflow, 'modules') && fromJSON('["self-hosted", "Linux", "X64", "modules"]') || contains(inputs.ci_workflow, 'firmware') && fromJSON('["self-hosted", "Linux", "X64", "firmware"]') || contains(inputs.ci_workflow, 'mm') && fromJSON('["self-hosted", "Linux", "X64", "mm"]') || fromJSON('["self-hosted"]') }}
+ needs: [setup, test, archive]
+ if: always()
+ steps:
+ - name: kdevops cleanup
+ uses: ./.github/actions/cleanup
+ with:
+ dir: ${{ inputs.ci_workflow }}
diff --git a/.github/workflows/manual.yml b/.github/workflows/manual.yml
new file mode 100644
index 00000000..30a035c8
--- /dev/null
+++ b/.github/workflows/manual.yml
@@ -0,0 +1,122 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Run kdevops CI Workflow - Manual
+
+on:
+ workflow_dispatch:
+ inputs:
+ ci_workflow:
+ description: "CI Workflow"
+ required: true
+ default: 'blktests_nvme'
+ type: choice
+ options:
+ - blktests
+ - blktests_block
+ - blktests_loop
+ - blktests_meta
+ - blktests_nbd
+ - blktests_nvme
+ - blktests_nvmemp
+ - blktests_scsi
+ - blktests_srp
+ - blktests_zbd
+ - lbs-xfs
+ - lbs-xfs-bdev-large-nvme
+ - lbs-xfs-bdev-nvme
+ - lbs-xfs-small
+ - xfs
+ - xfs_crc
+ - xfs_crc_logdev
+ - xfs_crc_rtdev
+ - xfs_crc_rtdev_extsize_28k
+ - xfs_crc_rtdev_extsize_64k
+ - xfs_nocrc
+ - xfs_nocrc_16k_4ks
+ - xfs_nocrc_1k
+ - xfs_nocrc_2k
+ - xfs_nocrc_32_4ks
+ - xfs_nocrc_4k
+ - xfs_nocrc_512
+ - xfs_nocrc_64_4ks
+ - xfs_nocrc_8k_4ks
+ - xfs_nocrc_lbs
+ - xfs_reflink
+ - xfs_reflink_1024
+ - xfs_reflink_16k_4ks
+ - xfs_reflink_2k
+ - xfs_reflink_32k_4ks
+ - xfs_reflink_4k
+ - xfs_reflink_4k_8ks
+ - xfs_reflink_64k_4ks
+ - xfs_reflink_dir_bsize_8k
+ - xfs_reflink_lbs
+ - xfs_reflink_logdev
+ - xfs_reflink_normapbt
+ - xfs_reflink_nrext64
+ - xfs_reflink_stripe_len
+ - xfs-soak
+ kernel_tree:
+ description: "Linux kernel tree to use"
+ required: true
+ default: 'linux'
+ type: choice
+ options:
+ - linux
+ - linux-next
+ - linux-stable
+ kernel_ref:
+ description: "Linux tree git reference (branch/tag/commit-id)"
+ required: true
+ default: 'master'
+ type: string
+ test_mode:
+ description: 'Testing mode'
+ required: false
+ default: 'full-testing'
+ type: choice
+ options:
+ - 'full-testing' # Complete test suite
+ - 'kdevops-validation' # Single test for framework validation
+ tests:
+ description: 'Custom test to run (for kdevops-validation mode only)'
+ required: false
+ type: string
+ default: ''
+
+jobs:
+ check_ref:
+ name: Check Linux kernel Git Reference
+ runs-on: [self-hosted]
+ steps:
+ - name: Checkout repo
+ uses: actions/checkout@v4
+
+ - name: Check kernel_ref exists
+ id: check_kernel_ref
+ run: |
+ set -euxo pipefail
+
+ ref="${{ github.event.inputs.kernel_ref }}"
+ tree="${{ github.event.inputs.kernel_tree }}"
+ mirror="/mirror/${tree}.git"
+ ls_remote="$(git ls-remote "$mirror" "refs/*/${ref}" || true)"
+ contains_branch="$(git -C "$mirror" branch --contains "${ref}" || true)"
+ contains_tag="$(git -C "$mirror" branch --contains "${ref}" || true)"
+
+ if [ -z "$ls_remote" ] && [ -z "$contains_branch" ] && [ -z "$contains_tag" ]; then
+ echo "Linux kernel ${ref} does not exist."
+ exit 1
+ fi
+
+ manual:
+ name: Manual kdevops CI
+ needs: [check_ref]
+ uses: ./.github/workflows/main.yml
+ secrets: inherit
+ with:
+ ci_workflow: ${{ inputs.ci_workflow }}
+ kernel_ref: ${{ inputs.kernel_ref }}
+ kernel_tree: ${{ inputs.kernel_tree }}
+ test_mode: ${{ inputs.test_mode }}
+ tests: ${{ inputs.tests }}
diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml
new file mode 100644
index 00000000..561e873e
--- /dev/null
+++ b/.github/workflows/push.yml
@@ -0,0 +1,29 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Run kdevops CI Workflow - Push/PR
+
+on:
+ push:
+ branches:
+ - '**'
+ pull_request:
+ branches:
+ - '**'
+
+jobs:
+ push-pr:
+ name: Push/PR kdevops CI
+ uses: ./.github/workflows/main.yml
+ secrets: inherit
+ strategy:
+ fail-fast: false # Don't cancel other jobs if one fails
+ matrix:
+ ci_workflow:
+ - blktests_nvme
+ - xfs_reflink_4k
+ with:
+ ci_workflow: ${{ matrix.ci_workflow }}
+ kernel_ref: 'v6.15'
+ kernel_tree: 'linux'
+ test_mode: 'kdevops-validation'
+ tests: ''
diff --git a/.github/workflows/schedule.yml b/.github/workflows/schedule.yml
new file mode 100644
index 00000000..8f4589b4
--- /dev/null
+++ b/.github/workflows/schedule.yml
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: GPL-2.0
+---
+name: Run kdevops CI Workflow - Schedule
+
+on:
+ schedule:
+ - cron: '0 14 * * *'
+
+ workflow_dispatch:
+
+jobs:
+ check_ref:
+ name: Check Linux kernel Git Reference
+ outputs:
+ kernel_ref: ${{ steps.check_kernel_ref.outputs.kernel_ref }}
+ kernel_tree: ${{ steps.check_kernel_ref.outputs.kernel_tree }}
+ runs-on: [self-hosted]
+ steps:
+ - name: Checkout repo
+ uses: actions/checkout@v4
+
+ - name: Generate kernel_ref
+ id: check_kernel_ref
+ run: |
+ set -euxo pipefail
+ day_of_week=$(date +%u)
+
+ kernel_ref=$(./scripts/korg-releases.py --moniker linux-next)
+ kernel_tree=linux-next
+ if [ "$day_of_week" -eq 1 ]; then
+ kernel_ref=$(./scripts/korg-releases.py --moniker mainline)
+ kernel_tree=linux
+ fi
+
+ "${{ github.workspace }}/scripts/github_output.sh" kernel_ref "$kernel_ref"
+ "${{ github.workspace }}/scripts/github_output.sh" kernel_tree "$kernel_tree"
+
+ schedule:
+ name: Scheduled kdevops CI
+ needs: [check_ref]
+ uses: ./.github/workflows/main.yml
+ secrets: inherit
+ strategy:
+ fail-fast: false # Don't cancel other jobs if one fails
+ matrix:
+ ci_workflow:
+ - blktests
+ - linux-mm-kpd
+ - linux-modules-kpd
+ - tmpfs
+ with:
+ ci_workflow: ${{ matrix.ci_workflow }}
+ kernel_ref: ${{ needs.check_ref.outputs.kernel_ref }}
+ kernel_tree: ${{ needs.check_ref.outputs.kernel_tree }}
diff --git a/scripts/generate_ci_commit_message.sh b/scripts/generate_ci_commit_message.sh
new file mode 100755
index 00000000..8be2c0b3
--- /dev/null
+++ b/scripts/generate_ci_commit_message.sh
@@ -0,0 +1,263 @@
+#!/bin/bash
+# SPDX-License-Identifier: copyleft-next-0.3.1
+
+# Generate enhanced CI commit message for kdevops-results-archive
+# This script creates commit messages following the kdevops CI format specification
+
+set -euo pipefail
+
+# Default values
+CI_WORKFLOW="${CI_WORKFLOW:-demo}"
+KERNEL_TREE="${KERNEL_TREE:-linux}"
+TESTS_PARAM="${TESTS:-${LIMIT_TESTS:-}}"
+DURATION="${DURATION:-unknown}"
+
+# Input files (expected to be created by GitHub Actions)
+CI_COMMIT_EXTRA="${CI_COMMIT_EXTRA:-ci.commit_extra}"
+CI_RESULT="${CI_RESULT:-ci.result}"
+CI_START_TIME="${CI_START_TIME:-ci.start_time}"
+CI_REF="${CI_REF:-ci.ref}"
+CI_TRIGGER="${CI_TRIGGER:-ci.trigger}"
+
+# Helper function to wrap long commit subjects at 72 chars
+wrap_commit_subject() {
+ local subject="$1"
+ if [ ${#subject} -le 68 ]; then # 68 to account for quotes
+ echo "\"$subject\""
+ else
+ echo "\"$subject\"" | fold -s -w 68 | sed '2,$s/^/ /'
+ fi
+}
+
+# Calculate duration if start time available
+calculate_duration() {
+ if [ -f "$CI_START_TIME" ]; then
+ local start_time=$(cat "$CI_START_TIME" 2>/dev/null || echo $(date +%s))
+ local end_time=$(date +%s)
+ local duration_sec=$((end_time - start_time))
+ local minutes=$((duration_sec / 60))
+ local seconds=$((duration_sec % 60))
+
+ if [ $minutes -gt 0 ]; then
+ echo "${minutes}m ${seconds}s"
+ else
+ echo "${seconds}s"
+ fi
+ else
+ echo "unknown"
+ fi
+}
+
+# Determine scope and header format with enhanced context-aware detection
+determine_scope() {
+ # Simple detection: if TESTS is set, it's kdevops validation
+ if [[ -n "${TESTS:-}" ]] || [[ -n "${LIMIT_TESTS:-}" ]] || [[ -n "${TESTS_PARAM:-}" ]]; then
+ echo "kdevops"
+ else
+ echo "tests"
+ fi
+}
+
+# Get kernel information
+get_kernel_info() {
+ # Try to get git describe from linux directory, fallback gracefully
+ local kernel_describe="unknown"
+ local kernel_hash="unknown"
+ local kernel_subject="unknown commit"
+
+ if [ -d "linux/.git" ]; then
+ kernel_describe=$(cd linux && git describe --tags --always --dirty 2>/dev/null || echo "unknown")
+ kernel_hash=$(cd linux && git rev-parse --short=12 HEAD 2>/dev/null || echo "unknown")
+ kernel_subject=$(cd linux && git log -1 --pretty=format:"%s" 2>/dev/null || echo "unknown commit")
+ fi
+
+ echo "$kernel_describe|$kernel_hash|$kernel_subject"
+}
+
+# Get kdevops information (from kdevops directory context)
+get_kdevops_info() {
+ # This will be run from the kdevops directory
+ local kdevops_hash=$(git rev-parse --short=12 HEAD 2>/dev/null || echo "unknown")
+ local kdevops_subject=$(git log -1 --pretty=format:"%s" 2>/dev/null || echo "unknown commit")
+
+ echo "$kdevops_hash|$kdevops_subject"
+}
+
+# Read test results
+get_test_results() {
+ local result_content=""
+ local test_result="unknown"
+
+ if [ -f "$CI_COMMIT_EXTRA" ]; then
+ result_content=$(cat "$CI_COMMIT_EXTRA")
+ else
+ result_content="No test results available."
+ fi
+
+ if [ -f "$CI_RESULT" ]; then
+ test_result=$(cat "$CI_RESULT" | tr -d '\n' | tr -d '\r')
+ fi
+
+ echo "$test_result|$result_content"
+}
+
+# Determine workflow type for result formatting
+get_workflow_type() {
+ case "$CI_WORKFLOW" in
+ *xfs*|*btrfs*|*ext4*|*tmpfs*|*lbs-xfs*) echo "fstests" ;;
+ blktests*) echo "blktests" ;;
+ selftests*|*firmware*|*modules*|*mm*) echo "selftests" ;;
+ ai_*) echo "ai" ;;
+ mmtests_*) echo "mmtests" ;;
+ ltp_*) echo "ltp" ;;
+ fio-tests*|fio_*) echo "fio-tests" ;;
+ minio_*) echo "minio" ;;
+ *) echo "other" ;;
+ esac
+}
+
+# Generate the enhanced commit message
+generate_commit_message() {
+ local scope=$(determine_scope)
+ local workflow_type=$(get_workflow_type)
+ local duration=$(calculate_duration)
+
+ # Get kernel tree and ref from metadata files
+ local actual_kernel_tree="$KERNEL_TREE"
+ local actual_kernel_ref="unknown"
+
+ if [ -f "$CI_TRIGGER" ]; then
+ actual_kernel_tree=$(cat "$CI_TRIGGER" | tr -d '\n' | tr -d '\r')
+ fi
+
+ if [ -f "$CI_REF" ]; then
+ actual_kernel_ref=$(cat "$CI_REF" | tr -d '\n' | tr -d '\r')
+ fi
+
+ # Get kernel info (this assumes we're in the kernel directory context)
+ local kernel_info=$(get_kernel_info)
+ local kernel_describe=$(echo "$kernel_info" | cut -d'|' -f1)
+ local kernel_hash=$(echo "$kernel_info" | cut -d'|' -f2)
+ local kernel_subject=$(echo "$kernel_info" | cut -d'|' -f3)
+
+ # Use metadata ref if available, fallback to git describe
+ # Ensure we use the most accurate kernel version consistently
+ if [ "$actual_kernel_ref" != "unknown" ] && [ "$actual_kernel_ref" != "" ]; then
+ kernel_describe="$actual_kernel_ref"
+ fi
+
+ # Debug kernel version information
+
+ # Get kdevops info (this assumes we can access kdevops directory)
+ local kdevops_info=$(get_kdevops_info)
+ local kdevops_hash=$(echo "$kdevops_info" | cut -d'|' -f1)
+ local kdevops_subject=$(echo "$kdevops_info" | cut -d'|' -f2)
+
+ # Get test results
+ local results_info=$(get_test_results)
+ local test_result=$(echo "$results_info" | cut -d'|' -f1)
+ local result_content=$(echo "$results_info" | cut -d'|' -f2-)
+
+ # Build header
+ local header=""
+ if [ "$scope" = "kdevops" ]; then
+ # kdevops-ci validation format: focus on kdevops commit being tested
+ header="kdevops-ci: $CI_WORKFLOW: $kdevops_hash $kdevops_subject"
+ else
+ # Full test suite format: include PASS/FAIL status
+ local status="PASS"
+ if [ "$test_result" = "not ok" ] || [ "$test_result" = "fail" ]; then
+ status="FAIL"
+ fi
+ header="$CI_WORKFLOW ($actual_kernel_tree $kernel_describe): $status"
+ fi
+
+ # Build scope description
+ local scope_desc=""
+ if [ "$scope" = "kdevops" ]; then
+ local test_param="${TESTS:-${LIMIT_TESTS:-${TESTS_PARAM:-}}}"
+ if [[ -n "$test_param" ]]; then
+ scope_desc="kdevops validation (single test: $test_param)"
+ else
+ scope_desc="kdevops validation"
+ fi
+ else
+ scope_desc="full test suite"
+ fi
+
+ # Wrap kernel commit subject
+ local wrapped_kernel_subject=$(wrap_commit_subject "$kernel_subject")
+ local wrapped_kdevops_subject=$(wrap_commit_subject "$kdevops_subject")
+
+ # Generate metadata line with 72-char handling
+ local metadata_line="workflow: $CI_WORKFLOW | tree: $actual_kernel_tree | ref: $kernel_describe | scope: $scope | result: $test_result"
+ local metadata_output=""
+ if [ ${#metadata_line} -gt 72 ]; then
+ metadata_output="workflow: $CI_WORKFLOW | tree: $actual_kernel_tree"$'\n'"ref: $kernel_describe | scope: $scope | result: $test_result"
+ else
+ metadata_output="$metadata_line"
+ fi
+
+ # Build kernel validation info
+ local kernel_info_line=""
+ if [ "$actual_kernel_ref" != "unknown" ] && [ "$kernel_describe" != "$actual_kernel_ref" ]; then
+ # Show both requested and actual if they differ
+ kernel_info_line=" Kernel: $actual_kernel_ref (requested) → $kernel_describe (actual)"
+ else
+ # Show just the kernel version if they match or no metadata
+ kernel_info_line=" Kernel: $kernel_describe"
+ fi
+
+ # Build the complete commit message based on scope
+ if [ "$scope" = "kdevops" ]; then
+ # kdevops-ci validation format
+ cat << EOF
+$header
+
+BUILD INFO:
+ kdevops: $kdevops_hash ($wrapped_kdevops_subject)
+ Workflow: $CI_WORKFLOW
+ Test: ${TESTS_PARAM:-"complete suite"}
+$kernel_info_line
+ Duration: $duration
+
+EXECUTION RESULTS:
+$result_content
+
+METADATA:
+workflow: $CI_WORKFLOW | scope: kdevops | test: ${TESTS_PARAM:-"complete suite"} | requested: $actual_kernel_ref | actual: $kernel_describe | result: $test_result
+EOF
+ else
+ # Full test suite format
+ cat << EOF
+$header
+
+BUILD INFO:
+$kernel_info_line ($wrapped_kernel_subject)
+ Workflow: $CI_WORKFLOW
+ kdevops: $kdevops_hash ($wrapped_kdevops_subject)
+ Scope: $scope_desc
+ Duration: $duration
+
+EXECUTION RESULTS:
+$result_content
+
+METADATA:
+$metadata_output | requested: $actual_kernel_ref | actual: $kernel_describe
+EOF
+ fi
+}
+
+# Main execution
+main() {
+ # Validate required environment
+ if [ -z "$CI_WORKFLOW" ]; then
+ echo "Error: CI_WORKFLOW environment variable is required" >&2
+ exit 1
+ fi
+
+ generate_commit_message
+}
+
+# Run main function
+main "$@"
\ No newline at end of file
diff --git a/scripts/github_output.sh b/scripts/github_output.sh
new file mode 100755
index 00000000..711306c3
--- /dev/null
+++ b/scripts/github_output.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+#
+# Usage: ./github_output.sh key value
+set -euxo pipefail
+
+key="$1"
+value="$2"
+
+echo "$key=$value" >> "$GITHUB_OUTPUT"
diff --git a/scripts/korg-releases.py b/scripts/korg-releases.py
new file mode 100755
index 00000000..de494327
--- /dev/null
+++ b/scripts/korg-releases.py
@@ -0,0 +1,94 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: copyleft-next-0.3.1
+import sys
+import logging
+import argparse
+import json
+import urllib.request
+import socket
+import re
+
+
+def parser():
+ parser = argparse.ArgumentParser(description="kernel.org/releases.json checker")
+ parser.add_argument("--debug", action="store_true", help="debug")
+ parser.add_argument(
+ "--moniker",
+ help="moniker (mainline, stable, longterm or linux-next)",
+ required=True,
+ )
+ parser.add_argument(
+ "--pname",
+ help="project name for User-Agent request",
+ default="kdevops",
+ )
+ parser.add_argument(
+ "--pversion",
+ help="project version for User-Agent request",
+ default="5.0.2",
+ )
+ return parser
+
+
+def _check_connection(host, port, timeout=2):
+ try:
+ with socket.create_connection((host, port), timeout):
+ logging.debug(f"Connection to {host} on port {port} succeeded!")
+ return True
+ except (socket.timeout, socket.error) as e:
+ logging.debug(f"Connection to {host} on port {port} failed: {e}")
+ return False
+
+
+def kreleases(args) -> None:
+ """Get the latest kernel releases from kernel.org/releases.json"""
+
+ reflist = []
+ if _check_connection("kernel.org", 80):
+ _url = "https://www.kernel.org/releases.json"
+ req = urllib.request.Request(
+ _url,
+ headers={
+ "User-Agent": f"{args.pname}/{args.pversion} (kdevops@lists.linux.dev)"
+ },
+ )
+ with urllib.request.urlopen(req) as url:
+ data = json.load(url)
+
+ for release in data["releases"]:
+ if release["moniker"] == args.moniker:
+ # Check if release.json is aa.bb.cc type
+ if re.compile(r"^\d+\.\d+(\.\d+|-rc\d+)?$").match(
+ release["version"]
+ ):
+ reflist.append("v" + release["version"])
+ else:
+ reflist.append(release["version"])
+
+ logging.debug(f"{reflist}")
+ for r in reflist:
+ print(r)
+
+
+def main() -> None:
+ """Kconfig choice generator for git refereces"""
+ log = logging.getLogger()
+ log.setLevel(logging.INFO)
+ p = parser()
+ args, _ = p.parse_known_args()
+ if args.debug:
+ log.setLevel(logging.DEBUG)
+
+ kreleases(args)
+
+
+if __name__ == "__main__":
+ ret = 0
+ try:
+ main()
+ except Exception:
+ ret = 1
+ import traceback
+
+ traceback.print_exc()
+ sys.exit(ret)
--
2.50.1
^ permalink raw reply related [flat|nested] 10+ messages in thread