* [PATCH dwarves 0/2] dwarves: Introduce github actions for CI
@ 2025-03-06 17:04 Alan Maguire
2025-03-06 17:04 ` [PATCH dwarves 1/2] dwarves: Add github actions to build, test Alan Maguire
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Alan Maguire @ 2025-03-06 17:04 UTC (permalink / raw)
To: acme
Cc: yonghong.song, dwarves, ast, andrii, bpf, song, eddyz87, olsajiri,
Alan Maguire
libbpf and bpf kernel patch infrastructure have made great use
of github actions to provide continuous integration (CI) testing.
Here the libbpf CI is adapted to build pahole and run the associated
selftests. Examples of what the action workflows look like are
at [1] and [2].
Details about the workflows can be found in patch 1.
Patch 2 fixes an issue exposed by the dwarves-build workflow -
a compilation error when building dwarves with clang.
[1] https://github.com/alan-maguire/dwarves/actions/runs/13588880188
[2] https://github.com/alan-maguire/dwarves/actions/runs/13588880200
Alan Maguire (2):
dwarves: Add github actions to build, test
dwarves: Fix clang warning about unused variable
.github/actions/debian/action.yml | 16 ++++++
.github/actions/setup/action.yml | 23 ++++++++
.github/workflows/build.yml | 37 ++++++++++++
.github/workflows/codeql.yml | 53 +++++++++++++++++
.github/workflows/coverity.yml | 33 +++++++++++
.github/workflows/lint.yml | 20 +++++++
.github/workflows/ondemand.yml | 31 ++++++++++
.github/workflows/test.yml | 36 ++++++++++++
.github/workflows/vmtest.yml | 94 +++++++++++++++++++++++++++++++
ci/managers/debian.sh | 88 +++++++++++++++++++++++++++++
ci/managers/travis_wait.bash | 61 ++++++++++++++++++++
dwarves_fprintf.c | 2 +-
12 files changed, 493 insertions(+), 1 deletion(-)
create mode 100644 .github/actions/debian/action.yml
create mode 100644 .github/actions/setup/action.yml
create mode 100644 .github/workflows/build.yml
create mode 100644 .github/workflows/codeql.yml
create mode 100644 .github/workflows/coverity.yml
create mode 100644 .github/workflows/lint.yml
create mode 100644 .github/workflows/ondemand.yml
create mode 100644 .github/workflows/test.yml
create mode 100644 .github/workflows/vmtest.yml
create mode 100755 ci/managers/debian.sh
create mode 100644 ci/managers/travis_wait.bash
--
2.43.5
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH dwarves 1/2] dwarves: Add github actions to build, test 2025-03-06 17:04 [PATCH dwarves 0/2] dwarves: Introduce github actions for CI Alan Maguire @ 2025-03-06 17:04 ` Alan Maguire 2025-03-12 20:28 ` Ihor Solodrai 2025-03-06 17:04 ` [PATCH dwarves 2/2] dwarves: Fix clang warning about unused variable Alan Maguire 2025-03-06 21:14 ` [PATCH dwarves 0/2] dwarves: Introduce github actions for CI Ihor Solodrai 2 siblings, 1 reply; 6+ messages in thread From: Alan Maguire @ 2025-03-06 17:04 UTC (permalink / raw) To: acme Cc: yonghong.song, dwarves, ast, andrii, bpf, song, eddyz87, olsajiri, Alan Maguire Borrowing heavily from libbpf github actions, add workflows to - build dwarves for gcc, LLVM - build dwarves for x86_64/aarch64 and use it to build a Linux kernel including BTF generation; then run dwarves selftests using generated vmlinux These workflows trigger on all pushes. This will alow both developers working on dwarves to push a branch to their github repo and test, and also for maintainer pushes from git.kernel.org pahole repo to trigger tests. And similar to libbpf, additional workflows for coverity etc are triggered for pushes to master/next. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> --- .github/actions/debian/action.yml | 16 ++++++ .github/actions/setup/action.yml | 23 ++++++++ .github/workflows/build.yml | 37 ++++++++++++ .github/workflows/codeql.yml | 53 +++++++++++++++++ .github/workflows/coverity.yml | 33 +++++++++++ .github/workflows/lint.yml | 20 +++++++ .github/workflows/ondemand.yml | 31 ++++++++++ .github/workflows/test.yml | 36 ++++++++++++ .github/workflows/vmtest.yml | 94 +++++++++++++++++++++++++++++++ ci/managers/debian.sh | 88 +++++++++++++++++++++++++++++ ci/managers/travis_wait.bash | 61 ++++++++++++++++++++ 11 files changed, 492 insertions(+) create mode 100644 .github/actions/debian/action.yml create mode 100644 .github/actions/setup/action.yml create mode 100644 .github/workflows/build.yml create mode 100644 .github/workflows/codeql.yml create mode 100644 .github/workflows/coverity.yml create mode 100644 .github/workflows/lint.yml create mode 100644 .github/workflows/ondemand.yml create mode 100644 .github/workflows/test.yml create mode 100644 .github/workflows/vmtest.yml create mode 100755 ci/managers/debian.sh create mode 100644 ci/managers/travis_wait.bash diff --git a/.github/actions/debian/action.yml b/.github/actions/debian/action.yml new file mode 100644 index 0000000..23d48ec --- /dev/null +++ b/.github/actions/debian/action.yml @@ -0,0 +1,16 @@ +name: 'debian' +description: 'Build' +inputs: + target: + description: 'Run target' + required: true +runs: + using: "composite" + steps: + - run: | + source /tmp/ci_setup + bash -x $CI_ROOT/managers/debian.sh SETUP + bash -x $CI_ROOT/managers/debian.sh ${{ inputs.target }} + bash -x $CI_ROOT/managers/debian.sh CLEANUP + shell: bash + diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml new file mode 100644 index 0000000..ae6eb78 --- /dev/null +++ b/.github/actions/setup/action.yml @@ -0,0 +1,23 @@ +name: 'setup' +description: 'setup env, create /tmp/ci_setup' +runs: + using: "composite" + steps: + - id: variables + run: | + export REPO_ROOT=$GITHUB_WORKSPACE + export CI_ROOT=$REPO_ROOT/ci + # this is somewhat ugly, but that is the easiest way to share this code with + # arch specific docker + echo 'echo ::group::Env setup' > /tmp/ci_setup + echo export DEBIAN_FRONTEND=noninteractive >> /tmp/ci_setup + echo sudo apt-get update >> /tmp/ci_setup + echo sudo apt-get install -y aptitude qemu-kvm zstd binutils-dev elfutils libcap-dev libelf-dev libdw-dev libguestfs-tools >> /tmp/ci_setup + echo export PROJECT_NAME='dwarves' >> /tmp/ci_setup + echo export AUTHOR_EMAIL="$(git log -1 --pretty=\"%aE\")" >> /tmp/ci_setup + echo export REPO_ROOT=$GITHUB_WORKSPACE >> /tmp/ci_setup + echo export CI_ROOT=$REPO_ROOT/ci >> /tmp/ci_setup + echo export VMTEST_ROOT=$CI_ROOT/vmtest >> /tmp/ci_setup + echo 'echo ::endgroup::' >> /tmp/ci_setup + shell: bash + diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml new file mode 100644 index 0000000..a95f71f --- /dev/null +++ b/.github/workflows/build.yml @@ -0,0 +1,37 @@ +name: dwarves-build + +on: + pull_request: + push: + schedule: + - cron: '0 18 * * *' + +concurrency: + group: ci-build-${{ github.head_ref }} + cancel-in-progress: true + +jobs: + + debian: + runs-on: ubuntu-latest + name: Debian Build (${{ matrix.name }}) + strategy: + fail-fast: false + matrix: + include: + - name: default + target: RUN + - name: gcc-12 + target: RUN_GCC12 + - name: clang + target: RUN_CLANG + steps: + - uses: actions/checkout@v4 + name: Checkout + - uses: ./.github/actions/setup + name: Setup + - uses: ./.github/actions/debian + name: Build + with: + target: ${{ matrix.target }} + diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml new file mode 100644 index 0000000..a140be1 --- /dev/null +++ b/.github/workflows/codeql.yml @@ -0,0 +1,53 @@ +--- +# vi: ts=2 sw=2 et: + +name: "CodeQL" + +on: + push: + branches: + - master + pull_request: + branches: + - master + - next + +permissions: + contents: read + +jobs: + analyze: + name: Analyze + runs-on: ubuntu-latest + concurrency: + group: ${{ github.workflow }}-${{ matrix.language }}-${{ github.ref }} + cancel-in-progress: true + permissions: + actions: read + security-events: write + + strategy: + fail-fast: false + matrix: + language: ['cpp', 'python'] + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Initialize CodeQL + uses: github/codeql-action/init@v2 + with: + languages: ${{ matrix.language }} + queries: +security-extended,security-and-quality + + - name: Setup + uses: ./.github/actions/setup + + - name: Build + run: | + source /tmp/ci_setup + make -C ./src + + - name: Perform CodeQL Analysis + uses: github/codeql-action/analyze@v2 diff --git a/.github/workflows/coverity.yml b/.github/workflows/coverity.yml new file mode 100644 index 0000000..c672a7d --- /dev/null +++ b/.github/workflows/coverity.yml @@ -0,0 +1,33 @@ +name: dwarves-ci-coverity + +on: + push: + branches: + - master + - next + schedule: + - cron: '0 18 * * *' + +jobs: + coverity: + runs-on: ubuntu-latest + name: Coverity + env: + COVERITY_SCAN_TOKEN: ${{ secrets.COVERITY_SCAN_TOKEN }} + steps: + - uses: actions/checkout@v4 + - uses: ./.github/actions/setup + - name: Run coverity + if: ${{ env.COVERITY_SCAN_TOKEN }} + run: | + source /tmp/ci_setup + export COVERITY_SCAN_NOTIFICATION_EMAIL="${AUTHOR_EMAIL}" + export COVERITY_SCAN_BRANCH_PATTERN=${GITHUB_REF##refs/*/} + export TRAVIS_BRANCH=${COVERITY_SCAN_BRANCH_PATTERN} + scripts/coverity.sh + env: + COVERITY_SCAN_PROJECT_NAME: dwarves + COVERITY_SCAN_BUILD_COMMAND_PREPEND: 'cmake .' + COVERITY_SCAN_BUILD_COMMAND: 'make' + - name: SCM log + run: cat /home/runner/work/dwarves/cov-int/scm_log.txt diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml new file mode 100644 index 0000000..ca13052 --- /dev/null +++ b/.github/workflows/lint.yml @@ -0,0 +1,20 @@ +name: "lint" + +on: + pull_request: + push: + branches: + - master + - next + +jobs: + shellcheck: + name: ShellCheck + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v4 + - name: Run ShellCheck + uses: ludeeus/action-shellcheck@master + env: + SHELLCHECK_OPTS: --severity=error diff --git a/.github/workflows/ondemand.yml b/.github/workflows/ondemand.yml new file mode 100644 index 0000000..5f3034f --- /dev/null +++ b/.github/workflows/ondemand.yml @@ -0,0 +1,31 @@ +name: ondemand + +on: + workflow_dispatch: + inputs: + arch: + default: 'x86_64' + required: true + llvm-version: + default: '18' + required: true + kernel: + default: 'LATEST' + required: true + pahole: + default: "master" + required: true + runs-on: + default: 'ubuntu-24.04' + required: true + +jobs: + vmtest: + name: ${{ inputs.kernel }} kernel llvm-${{ inputs.llvm-version }} pahole@${{ inputs.pahole }} + uses: ./.github/workflows/vmtest.yml + with: + runs_on: ${{ inputs.runs-on }} + kernel: ${{ inputs.kernel }} + arch: ${{ inputs.arch }} + llvm-version: ${{ inputs.llvm-version }} + pahole: ${{ inputs.pahole }} diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml new file mode 100644 index 0000000..f11ebfe --- /dev/null +++ b/.github/workflows/test.yml @@ -0,0 +1,36 @@ +name: dwarves-ci + +on: + pull_request: + push: + schedule: + - cron: '0 18 * * *' + +concurrency: + group: ci-test-${{ github.head_ref }} + cancel-in-progress: true + +jobs: + vmtest: + strategy: + fail-fast: false + matrix: + include: + - kernel: 'LATEST' + runs_on: 'ubuntu-24.04' + arch: 'x86_64' + llvm-version: '18' + pahole: 'master' + - kernel: 'LATEST' + runs_on: 'ubuntu-24.04-arm' + arch: 'aarch64' + llvm-version: '18' + pahole: 'tmp.master' + name: Linux ${{ matrix.kernel }} + uses: ./.github/workflows/vmtest.yml + with: + runs_on: ${{ matrix.runs_on }} + kernel: ${{ matrix.kernel }} + arch: ${{ matrix.arch }} + llvm-version: ${{ matrix.llvm-version }} + pahole: ${{ matrix.pahole }} diff --git a/.github/workflows/vmtest.yml b/.github/workflows/vmtest.yml new file mode 100644 index 0000000..aef5f0a --- /dev/null +++ b/.github/workflows/vmtest.yml @@ -0,0 +1,94 @@ +name: 'Build kernel run selftests via vmtest' + +on: + workflow_call: + inputs: + runs_on: + required: true + default: 'ubuntu-24.04' + type: string + arch: + description: 'what arch to test' + required: true + default: 'x86_64' + type: string + kernel: + description: 'kernel version or LATEST' + required: true + default: 'LATEST' + type: string + pahole: + description: 'pahole rev or branch' + required: false + default: 'master' + type: string + llvm-version: + description: 'llvm version' + required: false + default: '18' + type: string +jobs: + vmtest: + name: pahole@${{ inputs.arch }} + runs-on: ${{ inputs.runs_on }} + steps: + + - uses: actions/checkout@v4 + + - name: Setup environment + uses: libbpf/ci/setup-build-env@v3 + with: + pahole: ${{ inputs.pahole }} + arch: ${{ inputs.arch }} + llvm-version: ${{ inputs.llvm-version }} + + - name: Build,install current pahole + shell: bash + run: | + git config --global --add safe.directory ${{ github.workspace }} + git submodule update --init + mkdir build + cd build + cmake -DGIT_SUBMODULE=OFF -DBUILD_SHARED_LIBS=OFF .. + make -j$((4*$(nproc))) all + make DESTDIR=../install install + + - name: Get kernel source + uses: libbpf/ci/get-linux-source@v3 + with: + repo: 'https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git' + dest: '${{ github.workspace }}/.kernel' + + - name: Configure, build kernel with current pahole + shell: bash + run: | + export PATH=${{ github.workspace }}/install/usr/local/bin:${PATH} + export PAHOLE=${{ github.workspace }}/install/usr/local/bin/pahole + which pahole + $PAHOLE --version + cd .kernel + cat tools/testing/selftests/bpf/config \ + tools/testing/selftests/bpf/config.${{ inputs.arch }} > .config + # this file might or might not exist depending on kernel version + cat tools/testing/selftests/bpf/config.vm >> .config || : + make olddefconfig && make prepare + grep PAHOLE .config + grep _BTF .config + make -j $((4*$(nproc))) all + cp vmlinux ${{ github.workspace }} + cd - + + - name: Run selftests + env: + VMLINUX: ${{ github.workspace }}/vmlinux + LLVM_VERSION: ${{ inputs.llvm-version }} + SELFTESTS: ${{ github.workspace }}/tests + shell: bash + run: | + cd $SELFTESTS + export PATH=${{ github.workspace }}/install/usr/local/bin:${PATH} + which pahole + pahole --version + vmlinux=$VMLINUX ./tests + cd - + diff --git a/ci/managers/debian.sh b/ci/managers/debian.sh new file mode 100755 index 0000000..8316b02 --- /dev/null +++ b/ci/managers/debian.sh @@ -0,0 +1,88 @@ +#!/bin/bash + +PHASES=(${@:-SETUP RUN CLEANUP}) +DEBIAN_RELEASE="${DEBIAN_RELEASE:-testing}" +CONT_NAME="${CONT_NAME:-dwarves-debian-$DEBIAN_RELEASE}" +ENV_VARS="${ENV_VARS:-}" +DOCKER_RUN="${DOCKER_RUN:-docker run}" +REPO_ROOT="${REPO_ROOT:-$PWD}" +ADDITIONAL_DEPS=(pkgconf) +EXTRA_CFLAGS="" +EXTRA_LDFLAGS="" + +function info() { + echo -e "\033[33;1m$1\033[0m" +} + +function error() { + echo -e "\033[31;1m$1\033[0m" +} + +function docker_exec() { + docker exec $ENV_VARS $CONT_NAME "$@" +} + +set -eu + +source "$(dirname $0)/travis_wait.bash" + +for phase in "${PHASES[@]}"; do + case $phase in + SETUP) + info "Setup phase" + info "Using Debian $DEBIAN_RELEASE" + + docker --version + + docker pull debian:$DEBIAN_RELEASE + info "Starting container $CONT_NAME" + $DOCKER_RUN -v $REPO_ROOT:/build:rw \ + -w /build --privileged=true --name $CONT_NAME \ + -dit --net=host debian:$DEBIAN_RELEASE /bin/bash + echo -e "::group::Build Env Setup" + + docker_exec apt-get -y update + docker_exec apt-get -y install aptitude + docker_exec aptitude -y install make cmake libz-dev libelf-dev libdw-dev git + docker_exec aptitude -y install "${ADDITIONAL_DEPS[@]}" + echo -e "::endgroup::" + ;; + RUN|RUN_CLANG|RUN_CLANG16|RUN_GCC12) + CC="cc" + if [[ "$phase" =~ "RUN_CLANG(\d+)(_ASAN)?" ]]; then + ENV_VARS="-e CC=clang-${BASH_REMATCH[1]} -e CXX=clang++-${BASH_REMATCH[1]}" + CC="clang-${BASH_REMATCH[1]}" + elif [[ "$phase" = *"CLANG"* ]]; then + ENV_VARS="-e CC=clang -e CXX=clang++" + CC="clang" + elif [[ "$phase" =~ "RUN_GCC(\d+)(_ASAN)?" ]]; then + ENV_VARS="-e CC=gcc-${BASH_REMATCH[1]} -e CXX=g++-${BASH_REMATCH[1]}" + CC="gcc-${BASH_REMATCH[1]}" + fi + if [[ "$CC" != "cc" ]]; then + docker_exec aptitude -y install "$CC" + else + docker_exec aptitude -y install gcc + fi + git config --global --add safe.directory $REPO_ROOT + pushd $REPO_ROOT + git submodule update --init + popd + docker_exec mkdir build install + docker_exec ${CC} --version + info "build" + docker_exec cmake -DGIT_SUBMODULE=OFF . + docker_exec make -j$((4*$(nproc))) + info "install" + docker_exec make DESTDIR=../install install + ;; + CLEANUP) + info "Cleanup phase" + docker stop $CONT_NAME + docker rm -f $CONT_NAME + ;; + *) + echo >&2 "Unknown phase '$phase'" + exit 1 + esac +done diff --git a/ci/managers/travis_wait.bash b/ci/managers/travis_wait.bash new file mode 100644 index 0000000..acf6ad1 --- /dev/null +++ b/ci/managers/travis_wait.bash @@ -0,0 +1,61 @@ +# This was borrowed from https://github.com/travis-ci/travis-build/tree/master/lib/travis/build/bash +# to get around https://github.com/travis-ci/travis-ci/issues/9979. It should probably be removed +# as soon as Travis CI has started to provide an easy way to export the functions to bash scripts. + +travis_jigger() { + local cmd_pid="${1}" + shift + local timeout="${1}" + shift + local count=0 + + echo -e "\\n" + + while [[ "${count}" -lt "${timeout}" ]]; do + count="$((count + 1))" + echo -ne "Still running (${count} of ${timeout}): ${*}\\r" + sleep 60 + done + + echo -e "\\n${ANSI_RED}Timeout (${timeout} minutes) reached. Terminating \"${*}\"${ANSI_RESET}\\n" + kill -9 "${cmd_pid}" +} + +travis_wait() { + local timeout="${1}" + + if [[ "${timeout}" =~ ^[0-9]+$ ]]; then + shift + else + timeout=20 + fi + + local cmd=("${@}") + local log_file="travis_wait_${$}.log" + + "${cmd[@]}" &>"${log_file}" & + local cmd_pid="${!}" + + travis_jigger "${!}" "${timeout}" "${cmd[@]}" & + local jigger_pid="${!}" + local result + + { + set +e + wait "${cmd_pid}" 2>/dev/null + result="${?}" + ps -p"${jigger_pid}" &>/dev/null && kill "${jigger_pid}" + set -e + } + + if [[ "${result}" -eq 0 ]]; then + echo -e "\\n${ANSI_GREEN}The command ${cmd[*]} exited with ${result}.${ANSI_RESET}" + else + echo -e "\\n${ANSI_RED}The command ${cmd[*]} exited with ${result}.${ANSI_RESET}" + fi + + echo -e "\\n${ANSI_GREEN}Log:${ANSI_RESET}\\n" + cat "${log_file}" + + return "${result}" +} -- 2.43.5 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH dwarves 1/2] dwarves: Add github actions to build, test 2025-03-06 17:04 ` [PATCH dwarves 1/2] dwarves: Add github actions to build, test Alan Maguire @ 2025-03-12 20:28 ` Ihor Solodrai 0 siblings, 0 replies; 6+ messages in thread From: Ihor Solodrai @ 2025-03-12 20:28 UTC (permalink / raw) To: Alan Maguire, acme Cc: yonghong.song, dwarves, ast, andrii, bpf, song, eddyz87, olsajiri, Alan Maguire On 3/6/25 9:04 AM, Alan Maguire wrote: > Borrowing heavily from libbpf github actions, add workflows to > > - build dwarves for gcc, LLVM > > - build dwarves for x86_64/aarch64 and use it to build a Linux > kernel including BTF generation; then run dwarves selftests > using generated vmlinux > > These workflows trigger on all pushes. This will alow both > developers working on dwarves to push a branch to their github > repo and test, and also for maintainer pushes from git.kernel.org > pahole repo to trigger tests. > > And similar to libbpf, additional workflows for coverity etc > are triggered for pushes to master/next. > > Signed-off-by: Alan Maguire <alan.maguire@oracle.com> > --- > .github/actions/debian/action.yml | 16 ++++++ > .github/actions/setup/action.yml | 23 ++++++++ > .github/workflows/build.yml | 37 ++++++++++++ > .github/workflows/codeql.yml | 53 +++++++++++++++++ > .github/workflows/coverity.yml | 33 +++++++++++ > .github/workflows/lint.yml | 20 +++++++ > .github/workflows/ondemand.yml | 31 ++++++++++ > .github/workflows/test.yml | 36 ++++++++++++ > .github/workflows/vmtest.yml | 94 +++++++++++++++++++++++++++++++ > ci/managers/debian.sh | 88 +++++++++++++++++++++++++++++ > ci/managers/travis_wait.bash | 61 ++++++++++++++++++++ > 11 files changed, 492 insertions(+) > create mode 100644 .github/actions/debian/action.yml > create mode 100644 .github/actions/setup/action.yml > create mode 100644 .github/workflows/build.yml > create mode 100644 .github/workflows/codeql.yml > create mode 100644 .github/workflows/coverity.yml > create mode 100644 .github/workflows/lint.yml > create mode 100644 .github/workflows/ondemand.yml > create mode 100644 .github/workflows/test.yml > create mode 100644 .github/workflows/vmtest.yml > create mode 100755 ci/managers/debian.sh > create mode 100644 ci/managers/travis_wait.bash Hi Alan. See comments below. > > diff --git a/.github/actions/debian/action.yml b/.github/actions/debian/action.yml > new file mode 100644 > index 0000000..23d48ec > --- /dev/null > +++ b/.github/actions/debian/action.yml > @@ -0,0 +1,16 @@ > +name: 'debian' > +description: 'Build' > +inputs: > + target: > + description: 'Run target' > + required: true > +runs: > + using: "composite" > + steps: > + - run: | > + source /tmp/ci_setup > + bash -x $CI_ROOT/managers/debian.sh SETUP > + bash -x $CI_ROOT/managers/debian.sh ${{ inputs.target }} > + bash -x $CI_ROOT/managers/debian.sh CLEANUP > + shell: bash > + > diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml > new file mode 100644 > index 0000000..ae6eb78 > --- /dev/null > +++ b/.github/actions/setup/action.yml > @@ -0,0 +1,23 @@ > +name: 'setup' > +description: 'setup env, create /tmp/ci_setup' > +runs: > + using: "composite" > + steps: > + - id: variables > + run: | > + export REPO_ROOT=$GITHUB_WORKSPACE > + export CI_ROOT=$REPO_ROOT/ci > + # this is somewhat ugly, but that is the easiest way to share this code with > + # arch specific docker > + echo 'echo ::group::Env setup' > /tmp/ci_setup > + echo export DEBIAN_FRONTEND=noninteractive >> /tmp/ci_setup > + echo sudo apt-get update >> /tmp/ci_setup > + echo sudo apt-get install -y aptitude qemu-kvm zstd binutils-dev elfutils libcap-dev libelf-dev libdw-dev libguestfs-tools >> /tmp/ci_setup > + echo export PROJECT_NAME='dwarves' >> /tmp/ci_setup > + echo export AUTHOR_EMAIL="$(git log -1 --pretty=\"%aE\")" >> /tmp/ci_setup > + echo export REPO_ROOT=$GITHUB_WORKSPACE >> /tmp/ci_setup > + echo export CI_ROOT=$REPO_ROOT/ci >> /tmp/ci_setup > + echo export VMTEST_ROOT=$CI_ROOT/vmtest >> /tmp/ci_setup > + echo 'echo ::endgroup::' >> /tmp/ci_setup > + shell: bash > + I strongly suggest getting rid of local actions (`.github/actions` directory) and moving this into an independent shell script. Generally, it is much easier to develop and debug CI code if you can run it locally (say in a fresh docker container). From this follows that it is better to write most of it in separately executable scripts (in bash or python, like in bpftrace [1]), as opposed to inlining in yaml. For an example of how this might be done see very recently set up libbpf/usdt CI [2]. In this particular case, I'd put inlined code into ./ci/setup.sh and source it directly from bash when necesary. It's a little hypocritical of me to point this out, as libbpf still has all this mess. But I'd like you to avoid it while it's relatively simple. [1] https://github.com/bpftrace/bpftrace/blob/master/.github/include/ci.py [2] https://github.com/libbpf/usdt/tree/main/.github > diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml > new file mode 100644 > index 0000000..a95f71f > --- /dev/null > +++ b/.github/workflows/build.yml > @@ -0,0 +1,37 @@ > +name: dwarves-build > + > +on: > + pull_request: > + push: > + schedule: > + - cron: '0 18 * * *' > + > +concurrency: > + group: ci-build-${{ github.head_ref }} > + cancel-in-progress: true > + > +jobs: > + > + debian: > + runs-on: ubuntu-latest > + name: Debian Build (${{ matrix.name }}) > + strategy: > + fail-fast: false > + matrix: > + include: > + - name: default > + target: RUN > + - name: gcc-12 > + target: RUN_GCC12 > + - name: clang > + target: RUN_CLANG > + steps: > + - uses: actions/checkout@v4 > + name: Checkout > + - uses: ./.github/actions/setup > + name: Setup > + - uses: ./.github/actions/debian > + name: Build > + with: > + target: ${{ matrix.target }} > + > diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml > new file mode 100644 > index 0000000..a140be1 > --- /dev/null > +++ b/.github/workflows/codeql.yml > @@ -0,0 +1,53 @@ > +--- > +# vi: ts=2 sw=2 et: > + > +name: "CodeQL" > + > +on: > + push: > + branches: > + - master > + pull_request: > + branches: > + - master > + - next > + > +permissions: > + contents: read > + > +jobs: > + analyze: > + name: Analyze > + runs-on: ubuntu-latest > + concurrency: > + group: ${{ github.workflow }}-${{ matrix.language }}-${{ github.ref }} > + cancel-in-progress: true > + permissions: > + actions: read > + security-events: write > + > + strategy: > + fail-fast: false > + matrix: > + language: ['cpp', 'python'] > + > + steps: > + - name: Checkout repository > + uses: actions/checkout@v4 > + > + - name: Initialize CodeQL > + uses: github/codeql-action/init@v2 Supported codeql-action version is v3: https://github.com/github/codeql-action?tab=readme-ov-file#supported-versions-of-the-codeql-action > + with: > + languages: ${{ matrix.language }} > + queries: +security-extended,security-and-quality > + > + - name: Setup > + uses: ./.github/actions/setup > + > + - name: Build > + run: | > + source /tmp/ci_setup > + make -C ./src > + > + - name: Perform CodeQL Analysis > + uses: github/codeql-action/analyze@v2 Same: v3 here. > diff --git a/.github/workflows/coverity.yml b/.github/workflows/coverity.yml > new file mode 100644 > index 0000000..c672a7d > --- /dev/null > +++ b/.github/workflows/coverity.yml > @@ -0,0 +1,33 @@ > +name: dwarves-ci-coverity > + > +on: > + push: > + branches: > + - master > + - next > + schedule: > + - cron: '0 18 * * *' > + > +jobs: > + coverity: > + runs-on: ubuntu-latest > + name: Coverity > + env: > + COVERITY_SCAN_TOKEN: ${{ secrets.COVERITY_SCAN_TOKEN }} Assuming you're intentionally want to run coverity scans, you will have to set up COVERITY_SCAN_TOKEN secret variable in settings of the github repository. > + steps: > + - uses: actions/checkout@v4 > + - uses: ./.github/actions/setup > + - name: Run coverity > + if: ${{ env.COVERITY_SCAN_TOKEN }} > + run: | > + source /tmp/ci_setup > + export COVERITY_SCAN_NOTIFICATION_EMAIL="${AUTHOR_EMAIL}" > + export COVERITY_SCAN_BRANCH_PATTERN=${GITHUB_REF##refs/*/} > + export TRAVIS_BRANCH=${COVERITY_SCAN_BRANCH_PATTERN} > + scripts/coverity.sh > + env: > + COVERITY_SCAN_PROJECT_NAME: dwarves > + COVERITY_SCAN_BUILD_COMMAND_PREPEND: 'cmake .' > + COVERITY_SCAN_BUILD_COMMAND: 'make' > + - name: SCM log > + run: cat /home/runner/work/dwarves/cov-int/scm_log.txt > diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml > new file mode 100644 > index 0000000..ca13052 > --- /dev/null > +++ b/.github/workflows/lint.yml > @@ -0,0 +1,20 @@ > +name: "lint" > + > +on: > + pull_request: > + push: > + branches: > + - master > + - next > + > +jobs: > + shellcheck: > + name: ShellCheck > + runs-on: ubuntu-latest > + steps: > + - name: Checkout repository > + uses: actions/checkout@v4 > + - name: Run ShellCheck > + uses: ludeeus/action-shellcheck@master > + env: > + SHELLCHECK_OPTS: --severity=error > diff --git a/.github/workflows/ondemand.yml b/.github/workflows/ondemand.yml > new file mode 100644 > index 0000000..5f3034f > --- /dev/null > +++ b/.github/workflows/ondemand.yml The usefullness of ondemand is dubious, tbh. Usually you want to run CI against a change in the code, which can be achieved by enabling testing workflows on pull_request. IMO ondemand workflow is not necessary. > @@ -0,0 +1,31 @@ > +name: ondemand > + > +on: > + workflow_dispatch: > + inputs: > + arch: > + default: 'x86_64' > + required: true > + llvm-version: > + default: '18' > + required: true > + kernel: > + default: 'LATEST' > + required: true > + pahole: > + default: "master" > + required: true > + runs-on: > + default: 'ubuntu-24.04' > + required: true > + > +jobs: > + vmtest: > + name: ${{ inputs.kernel }} kernel llvm-${{ inputs.llvm-version }} pahole@${{ inputs.pahole }} > + uses: ./.github/workflows/vmtest.yml > + with: > + runs_on: ${{ inputs.runs-on }} > + kernel: ${{ inputs.kernel }} > + arch: ${{ inputs.arch }} > + llvm-version: ${{ inputs.llvm-version }} > + pahole: ${{ inputs.pahole }} > diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml > new file mode 100644 > index 0000000..f11ebfe > --- /dev/null > +++ b/.github/workflows/test.yml > @@ -0,0 +1,36 @@ > +name: dwarves-ci > + > +on: > + pull_request: > + push: > + schedule: > + - cron: '0 18 * * *' > + > +concurrency: > + group: ci-test-${{ github.head_ref }} > + cancel-in-progress: true > + > +jobs: > + vmtest: > + strategy: > + fail-fast: false > + matrix: > + include: > + - kernel: 'LATEST' > + runs_on: 'ubuntu-24.04' > + arch: 'x86_64' > + llvm-version: '18' > + pahole: 'master' > + - kernel: 'LATEST' > + runs_on: 'ubuntu-24.04-arm' > + arch: 'aarch64' > + llvm-version: '18' > + pahole: 'tmp.master' > + name: Linux ${{ matrix.kernel }} > + uses: ./.github/workflows/vmtest.yml > + with: > + runs_on: ${{ matrix.runs_on }} > + kernel: ${{ matrix.kernel }} > + arch: ${{ matrix.arch }} > + llvm-version: ${{ matrix.llvm-version }} > + pahole: ${{ matrix.pahole }} llvm-version and pahole inputs don't make any sense for dwarves testing. These are inputs of the libbpf/ci/setup-build-env, because selftests/bpf need LLVM, and BPF CI also builds kernel with LLVM (do you want to do that too for pahole testing?). My point is, you may be able to avoid the dependency on libbpf/ci/setup-build-env and simply install required packages directly. It'll be less work for a job, and these inputs won't be necessary. > diff --git a/.github/workflows/vmtest.yml b/.github/workflows/vmtest.yml > new file mode 100644 > index 0000000..aef5f0a > --- /dev/null > +++ b/.github/workflows/vmtest.yml > @@ -0,0 +1,94 @@ > +name: 'Build kernel run selftests via vmtest' > + > +on: > + workflow_call: > + inputs: > + runs_on: > + required: true > + default: 'ubuntu-24.04' > + type: string > + arch: > + description: 'what arch to test' > + required: true > + default: 'x86_64' > + type: string > + kernel: > + description: 'kernel version or LATEST' > + required: true > + default: 'LATEST' > + type: string > + pahole: > + description: 'pahole rev or branch' > + required: false > + default: 'master' > + type: string > + llvm-version: > + description: 'llvm version' > + required: false > + default: '18' > + type: string > +jobs: > + vmtest: > + name: pahole@${{ inputs.arch }} > + runs-on: ${{ inputs.runs_on }} > + steps: > + > + - uses: actions/checkout@v4 > + > + - name: Setup environment > + uses: libbpf/ci/setup-build-env@v3 > + with: > + pahole: ${{ inputs.pahole }} > + arch: ${{ inputs.arch }} > + llvm-version: ${{ inputs.llvm-version }} > + > + - name: Build,install current pahole > + shell: bash > + run: | > + git config --global --add safe.directory ${{ github.workspace }} > + git submodule update --init > + mkdir build > + cd build > + cmake -DGIT_SUBMODULE=OFF -DBUILD_SHARED_LIBS=OFF .. > + make -j$((4*$(nproc))) all > + make DESTDIR=../install install > + > + - name: Get kernel source > + uses: libbpf/ci/get-linux-source@v3 > + with: > + repo: 'https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git' > + dest: '${{ github.workspace }}/.kernel' > + > + - name: Configure, build kernel with current pahole > + shell: bash > + run: | > + export PATH=${{ github.workspace }}/install/usr/local/bin:${PATH} > + export PAHOLE=${{ github.workspace }}/install/usr/local/bin/pahole > + which pahole > + $PAHOLE --version > + cd .kernel > + cat tools/testing/selftests/bpf/config \ > + tools/testing/selftests/bpf/config.${{ inputs.arch }} > .config > + # this file might or might not exist depending on kernel version > + cat tools/testing/selftests/bpf/config.vm >> .config || : > + make olddefconfig && make prepare > + grep PAHOLE .config > + grep _BTF .config > + make -j $((4*$(nproc))) all > + cp vmlinux ${{ github.workspace }} > + cd - > + > + - name: Run selftests nit: probably worth giving this step a different name. > + env: > + VMLINUX: ${{ github.workspace }}/vmlinux > + LLVM_VERSION: ${{ inputs.llvm-version }} > + SELFTESTS: ${{ github.workspace }}/tests > + shell: bash > + run: | > + cd $SELFTESTS > + export PATH=${{ github.workspace }}/install/usr/local/bin:${PATH} > + which pahole > + pahole --version > + vmlinux=$VMLINUX ./tests > + cd - > + > diff --git a/ci/managers/debian.sh b/ci/managers/debian.sh This script is too complicated for my taste. I would replace it with something like docker run step + build-in-docker.sh combination in libbpf for ubuntu: * https://github.com/libbpf/libbpf/blob/master/.github/workflows/build.yml#L85-L91 * https://github.com/libbpf/libbpf/blob/master/ci/build-in-docker.sh There are more potential parameters here, but all this logic with phases is just redundant IMO. For example, apt packages installation can be covered by a single shared script (that assumes ubuntu/debian). > new file mode 100755 > index 0000000..8316b02 > --- /dev/null > +++ b/ci/managers/debian.sh > @@ -0,0 +1,88 @@ > +#!/bin/bash > + > +PHASES=(${@:-SETUP RUN CLEANUP}) > +DEBIAN_RELEASE="${DEBIAN_RELEASE:-testing}" > +CONT_NAME="${CONT_NAME:-dwarves-debian-$DEBIAN_RELEASE}" > +ENV_VARS="${ENV_VARS:-}" > +DOCKER_RUN="${DOCKER_RUN:-docker run}" > +REPO_ROOT="${REPO_ROOT:-$PWD}" > +ADDITIONAL_DEPS=(pkgconf) > +EXTRA_CFLAGS="" > +EXTRA_LDFLAGS="" > + > +function info() { > + echo -e "\033[33;1m$1\033[0m" > +} > + > +function error() { > + echo -e "\033[31;1m$1\033[0m" > +} > + > +function docker_exec() { > + docker exec $ENV_VARS $CONT_NAME "$@" > +} > + > +set -eu > + > +source "$(dirname $0)/travis_wait.bash" > + > +for phase in "${PHASES[@]}"; do > + case $phase in > + SETUP) > + info "Setup phase" > + info "Using Debian $DEBIAN_RELEASE" > + > + docker --version > + > + docker pull debian:$DEBIAN_RELEASE > + info "Starting container $CONT_NAME" > + $DOCKER_RUN -v $REPO_ROOT:/build:rw \ > + -w /build --privileged=true --name $CONT_NAME \ > + -dit --net=host debian:$DEBIAN_RELEASE /bin/bash > + echo -e "::group::Build Env Setup" > + > + docker_exec apt-get -y update > + docker_exec apt-get -y install aptitude > + docker_exec aptitude -y install make cmake libz-dev libelf-dev libdw-dev git > + docker_exec aptitude -y install "${ADDITIONAL_DEPS[@]}" > + echo -e "::endgroup::" > + ;; > + RUN|RUN_CLANG|RUN_CLANG16|RUN_GCC12) > + CC="cc" > + if [[ "$phase" =~ "RUN_CLANG(\d+)(_ASAN)?" ]]; then > + ENV_VARS="-e CC=clang-${BASH_REMATCH[1]} -e CXX=clang++-${BASH_REMATCH[1]}" > + CC="clang-${BASH_REMATCH[1]}" > + elif [[ "$phase" = *"CLANG"* ]]; then > + ENV_VARS="-e CC=clang -e CXX=clang++" > + CC="clang" > + elif [[ "$phase" =~ "RUN_GCC(\d+)(_ASAN)?" ]]; then > + ENV_VARS="-e CC=gcc-${BASH_REMATCH[1]} -e CXX=g++-${BASH_REMATCH[1]}" > + CC="gcc-${BASH_REMATCH[1]}" > + fi > + if [[ "$CC" != "cc" ]]; then > + docker_exec aptitude -y install "$CC" > + else > + docker_exec aptitude -y install gcc > + fi > + git config --global --add safe.directory $REPO_ROOT > + pushd $REPO_ROOT > + git submodule update --init > + popd > + docker_exec mkdir build install > + docker_exec ${CC} --version > + info "build" > + docker_exec cmake -DGIT_SUBMODULE=OFF . > + docker_exec make -j$((4*$(nproc))) > + info "install" > + docker_exec make DESTDIR=../install install > + ;; > + CLEANUP) > + info "Cleanup phase" > + docker stop $CONT_NAME > + docker rm -f $CONT_NAME > + ;; > + *) > + echo >&2 "Unknown phase '$phase'" > + exit 1 > + esac > +done > diff --git a/ci/managers/travis_wait.bash b/ci/managers/travis_wait.bash > new file mode 100644 > index 0000000..acf6ad1 > --- /dev/null > +++ b/ci/managers/travis_wait.bash I am not sure you need this script. I see it's used in ci/managers/debian.sh, but I don't know it's purpose. > @@ -0,0 +1,61 @@ > +# This was borrowed from https://github.com/travis-ci/travis-build/tree/master/lib/travis/build/bash > +# to get around https://github.com/travis-ci/travis-ci/issues/9979. It should probably be removed > +# as soon as Travis CI has started to provide an easy way to export the functions to bash scripts. > + > +travis_jigger() { > + local cmd_pid="${1}" > + shift > + local timeout="${1}" > + shift > + local count=0 > + > + echo -e "\\n" > + > + while [[ "${count}" -lt "${timeout}" ]]; do > + count="$((count + 1))" > + echo -ne "Still running (${count} of ${timeout}): ${*}\\r" > + sleep 60 > + done > + > + echo -e "\\n${ANSI_RED}Timeout (${timeout} minutes) reached. Terminating \"${*}\"${ANSI_RESET}\\n" > + kill -9 "${cmd_pid}" > +} > + > +travis_wait() { > + local timeout="${1}" > + > + if [[ "${timeout}" =~ ^[0-9]+$ ]]; then > + shift > + else > + timeout=20 > + fi > + > + local cmd=("${@}") > + local log_file="travis_wait_${$}.log" > + > + "${cmd[@]}" &>"${log_file}" & > + local cmd_pid="${!}" > + > + travis_jigger "${!}" "${timeout}" "${cmd[@]}" & > + local jigger_pid="${!}" > + local result > + > + { > + set +e > + wait "${cmd_pid}" 2>/dev/null > + result="${?}" > + ps -p"${jigger_pid}" &>/dev/null && kill "${jigger_pid}" > + set -e > + } > + > + if [[ "${result}" -eq 0 ]]; then > + echo -e "\\n${ANSI_GREEN}The command ${cmd[*]} exited with ${result}.${ANSI_RESET}" > + else > + echo -e "\\n${ANSI_RED}The command ${cmd[*]} exited with ${result}.${ANSI_RESET}" > + fi > + > + echo -e "\\n${ANSI_GREEN}Log:${ANSI_RESET}\\n" > + cat "${log_file}" > + > + return "${result}" > +} ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH dwarves 2/2] dwarves: Fix clang warning about unused variable 2025-03-06 17:04 [PATCH dwarves 0/2] dwarves: Introduce github actions for CI Alan Maguire 2025-03-06 17:04 ` [PATCH dwarves 1/2] dwarves: Add github actions to build, test Alan Maguire @ 2025-03-06 17:04 ` Alan Maguire 2025-03-06 21:14 ` [PATCH dwarves 0/2] dwarves: Introduce github actions for CI Ihor Solodrai 2 siblings, 0 replies; 6+ messages in thread From: Alan Maguire @ 2025-03-06 17:04 UTC (permalink / raw) To: acme Cc: yonghong.song, dwarves, ast, andrii, bpf, song, eddyz87, olsajiri, Alan Maguire With dwarves CI, clang builds give a legit warning: /build/dwarves_fprintf.c:2102:9: error: variable 'printed' set but not used [-Werror,-Wunused-but-set-variable] 2102 | size_t printed = fprintf(fp, "namespace %s {\n", namespace__name(space)); | ^ 1 error generated. And in fact we accumulate values in printed but never use it for the return value. Add the printed count to the final fprintf(). Signed-off-by: Alan Maguire <alan.maguire@oracle.com> --- dwarves_fprintf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dwarves_fprintf.c b/dwarves_fprintf.c index c3e7f3c..4407fd1 100644 --- a/dwarves_fprintf.c +++ b/dwarves_fprintf.c @@ -2110,7 +2110,7 @@ static size_t namespace__fprintf(const struct tag *tag, const struct cu *cu, printed += fprintf(fp, "\n\n"); } - return fprintf(fp, "%.*s}", conf->indent, tabs); + return printed + fprintf(fp, "%.*s}", conf->indent, tabs); } size_t tag__fprintf(struct tag *tag, const struct cu *cu, -- 2.43.5 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH dwarves 0/2] dwarves: Introduce github actions for CI 2025-03-06 17:04 [PATCH dwarves 0/2] dwarves: Introduce github actions for CI Alan Maguire 2025-03-06 17:04 ` [PATCH dwarves 1/2] dwarves: Add github actions to build, test Alan Maguire 2025-03-06 17:04 ` [PATCH dwarves 2/2] dwarves: Fix clang warning about unused variable Alan Maguire @ 2025-03-06 21:14 ` Ihor Solodrai [not found] ` <CA+JHD929w2i8TsEZUHqoMsmJ20-OF5C6ytHng0UobN2W0ptmag@mail.gmail.com> 2 siblings, 1 reply; 6+ messages in thread From: Ihor Solodrai @ 2025-03-06 21:14 UTC (permalink / raw) To: Alan Maguire, acme Cc: yonghong.song, dwarves, ast, andrii, bpf, song, eddyz87, olsajiri, Alan Maguire On 3/6/25 9:04 AM, Alan Maguire wrote: > libbpf and bpf kernel patch infrastructure have made great use > of github actions to provide continuous integration (CI) testing. > Here the libbpf CI is adapted to build pahole and run the associated > selftests. Examples of what the action workflows look like are > at [1] and [2]. > > Details about the workflows can be found in patch 1. > > Patch 2 fixes an issue exposed by the dwarves-build workflow - > a compilation error when building dwarves with clang. > > > [1] https://github.com/alan-maguire/dwarves/actions/runs/13588880188 > [2] https://github.com/alan-maguire/dwarves/actions/runs/13588880200 Hi Alan. This is great! Glad to see you're working on it. I haven't read through the changes yet, but I already see that most of the CI code was copied from libbpf. Just want to note that you might not want to reproduce all the workflows from there in dwarves. And also there are inconveniences with local actions and ci/managers etc. I think it's worth it to try and eliminate as much of that code as possible, given you're starting from a blank slate. If you haven't done so already, you might want to check out "pahole staging" job that I tried on BPF CI infrastructure some time ago: https://github.com/kernel-patches/vmtest/pull/330/files It's a bit different from libbpf, as it reuses BPF CI workflows. But you might get some ideas there about simplifying dwarves CI. Another question is: are you sure about merging CI code upstream? Both for libbpf and kernel-patches/bpf the CI code lives independently of upstream and is synced from time to time on github. My guess is, it's because .github code is unlikely to get merged into the main Linux tree (which also makes sense). > > Alan Maguire (2): > dwarves: Add github actions to build, test > dwarves: Fix clang warning about unused variable > > .github/actions/debian/action.yml | 16 ++++++ > .github/actions/setup/action.yml | 23 ++++++++ > .github/workflows/build.yml | 37 ++++++++++++ > .github/workflows/codeql.yml | 53 +++++++++++++++++ > .github/workflows/coverity.yml | 33 +++++++++++ > .github/workflows/lint.yml | 20 +++++++ > .github/workflows/ondemand.yml | 31 ++++++++++ > .github/workflows/test.yml | 36 ++++++++++++ > .github/workflows/vmtest.yml | 94 +++++++++++++++++++++++++++++++ > ci/managers/debian.sh | 88 +++++++++++++++++++++++++++++ > ci/managers/travis_wait.bash | 61 ++++++++++++++++++++ > dwarves_fprintf.c | 2 +- > 12 files changed, 493 insertions(+), 1 deletion(-) > create mode 100644 .github/actions/debian/action.yml > create mode 100644 .github/actions/setup/action.yml > create mode 100644 .github/workflows/build.yml > create mode 100644 .github/workflows/codeql.yml > create mode 100644 .github/workflows/coverity.yml > create mode 100644 .github/workflows/lint.yml > create mode 100644 .github/workflows/ondemand.yml > create mode 100644 .github/workflows/test.yml > create mode 100644 .github/workflows/vmtest.yml > create mode 100755 ci/managers/debian.sh > create mode 100644 ci/managers/travis_wait.bash > ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <CA+JHD929w2i8TsEZUHqoMsmJ20-OF5C6ytHng0UobN2W0ptmag@mail.gmail.com>]
* Re: [PATCH dwarves 0/2] dwarves: Introduce github actions for CI [not found] ` <CA+JHD929w2i8TsEZUHqoMsmJ20-OF5C6ytHng0UobN2W0ptmag@mail.gmail.com> @ 2025-03-12 16:50 ` Alan Maguire 0 siblings, 0 replies; 6+ messages in thread From: Alan Maguire @ 2025-03-12 16:50 UTC (permalink / raw) To: Arnaldo Carvalho de Melo, Ihor Solodrai Cc: Arnaldo Carvalho de Melo, Yonghong Song, dwarves, Alexei Starovoitov, Andrii Nakryiko, bpf, Song Liu, Eddy Z, Jiri Olsa On 06/03/2025 23:45, Arnaldo Carvalho de Melo wrote: > > > On Thu, Mar 6, 2025, 6:14 PM Ihor Solodrai <ihor.solodrai@linux.dev > <mailto:ihor.solodrai@linux.dev>> wrote: > > On 3/6/25 9:04 AM, Alan Maguire wrote: > > libbpf and bpf kernel patch infrastructure have made great use > > of github actions to provide continuous integration (CI) testing. > > Here the libbpf CI is adapted to build pahole and run the associated > > selftests. Examples of what the action workflows look like are > > at [1] and [2]. > > > > Details about the workflows can be found in patch 1. > > > > Patch 2 fixes an issue exposed by the dwarves-build workflow - > > a compilation error when building dwarves with clang. > > > > > > [1] https://github.com/alan-maguire/dwarves/actions/ > runs/13588880188 <https://github.com/alan-maguire/dwarves/actions/ > runs/13588880188> > > [2] https://github.com/alan-maguire/dwarves/actions/ > runs/13588880200 <https://github.com/alan-maguire/dwarves/actions/ > runs/13588880200> > > Hi Alan. This is great! Glad to see you're working on it. > > > Indeed, having these tests in place is a really great development. > > > I haven't read through the changes yet, but I already see that most of > the CI code was copied from libbpf. Just want to note that you might > not want to reproduce all the workflows from there in dwarves. And > also there are inconveniences with local actions and ci/managers > etc. I think it's worth it to try and eliminate as much of that code > as possible, given you're starting from a blank slate. > > > If you haven't done so already, you might want to check out "pahole > staging" job that I tried on BPF CI infrastructure some time ago: > https://github.com/kernel-patches/vmtest/pull/330/files <https:// > github.com/kernel-patches/vmtest/pull/330/files> > > It's a bit different from libbpf, as it reuses BPF CI workflows. But > you might get some ideas there about simplifying dwarves CI. > Hi Ihor, There is a bit of redundancy in the way things work currently alright in the vmtest workflow [1] since it uses libbpf/ci/setup-build-env@v3 to set up LLVM etc and also builds pahole. That latter step isn't needed as we are building pahole ourselves later on in the workflow in order to test the current changes. But aside from that the only other libbpf ci operation is getting the linux source via libbpf/ci/get-linux-source@v3. In the case of the dwarves/pahole workflow I wanted to concentrate on the steps that I normally do when testing changes: build pahole for x86_64, aarch64, build a bpf-next kernel with it, and run the dwarves selftests. BPF selftests are already covered by BPF CI and thanks to your work we get pahole testing in BPF CI when changes land so that's great! The build workflow could perhaps be simplified a bit, but I wanted to preserve the value of having a matrix of toolchains (clang/gcc). There may be more efficient ways to achieve that; hopefully we can evolve the workflow to be more succinct in the future. > > > I'll try to take a look as well. > > > Another question is: are you sure about merging CI code upstream? Both > for libbpf and kernel-patches/bpf the CI code lives independently of > upstream and is synced from time to time on github. My guess is, it's > because .github code is unlikely to get merged into the main Linux > tree (which also makes sense). > > > I don't see a problem on having it in the pahole main repository, to > stimulate adding new tests while developing new features, no? > Yeah I think we're less constrained as we're not a kernel tree, we are just hosted on git.kernel.org. Arnaldo rightly pointed out we should have some instructions for how to trigger actions and use them to test changes; I'll wait a bit for additional comments etc and send a v2 incorporating that (into the toplevel README perhaps?). Thanks! Alan > - Arnaldo > > From smartphone > > > > > > Alan Maguire (2): > > dwarves: Add github actions to build, test > > dwarves: Fix clang warning about unused variable > > > > .github/actions/debian/action.yml | 16 ++++++ > > .github/actions/setup/action.yml | 23 ++++++++ > > .github/workflows/build.yml | 37 ++++++++++++ > > .github/workflows/codeql.yml | 53 +++++++++++++++++ > > .github/workflows/coverity.yml | 33 +++++++++++ > > .github/workflows/lint.yml | 20 +++++++ > > .github/workflows/ondemand.yml | 31 ++++++++++ > > .github/workflows/test.yml | 36 ++++++++++++ > > .github/workflows/vmtest.yml | 94 ++++++++++++++++++++++++++ > +++++ > > ci/managers/debian.sh | 88 +++++++++++++++++++++++++++++ > > ci/managers/travis_wait.bash | 61 ++++++++++++++++++++ > > dwarves_fprintf.c | 2 +- > > 12 files changed, 493 insertions(+), 1 deletion(-) > > create mode 100644 .github/actions/debian/action.yml > > create mode 100644 .github/actions/setup/action.yml > > create mode 100644 .github/workflows/build.yml > > create mode 100644 .github/workflows/codeql.yml > > create mode 100644 .github/workflows/coverity.yml > > create mode 100644 .github/workflows/lint.yml > > create mode 100644 .github/workflows/ondemand.yml > > create mode 100644 .github/workflows/test.yml > > create mode 100644 .github/workflows/vmtest.yml > > create mode 100755 ci/managers/debian.sh > > create mode 100644 ci/managers/travis_wait.bash > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-03-12 20:28 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-03-06 17:04 [PATCH dwarves 0/2] dwarves: Introduce github actions for CI Alan Maguire
2025-03-06 17:04 ` [PATCH dwarves 1/2] dwarves: Add github actions to build, test Alan Maguire
2025-03-12 20:28 ` Ihor Solodrai
2025-03-06 17:04 ` [PATCH dwarves 2/2] dwarves: Fix clang warning about unused variable Alan Maguire
2025-03-06 21:14 ` [PATCH dwarves 0/2] dwarves: Introduce github actions for CI Ihor Solodrai
[not found] ` <CA+JHD929w2i8TsEZUHqoMsmJ20-OF5C6ytHng0UobN2W0ptmag@mail.gmail.com>
2025-03-12 16:50 ` Alan Maguire
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox