[PATCH v2 dwarves 0/2] Add function list comparison to github CI

[PATCH v2 dwarves 0/2] Add function list comparison to github CI

[Alan Maguire]

As changes to pahole can generate different sets of functions for
vmlinux BTF, automate comparison between next branch and the series
CI is testing.  We already build default pahole so use it to baseline
function generation in BTF and use pfunct to generate function lists.
Example job here [1].

Patch 1 ensures that the baseline pahole builds the "next" branch
version; patch 2 adds the compare-functions.sh job to the dwarves-ci
pipeline.

Changes since v1:

- fix rebuild to actually rebuild with baseline pahole (patch 2)

[1] https://github.com/alan-maguire/dwarves/actions/runs/15872520906

Alan Maguire (2):
  github CI: Build pahole "next" branch as well as latest changes
  github CI: Add comparison of generated BTF functions between baseline,
    change

 .github/scripts/compare-functions.sh | 30 ++++++++++++++++++++++++++++
 .github/workflows/test.yml           |  4 ++--
 .github/workflows/vmtest.yml         |  4 ++++
 3 files changed, 36 insertions(+), 2 deletions(-)
 create mode 100755 .github/scripts/compare-functions.sh

-- 
2.43.5

[PATCH v2 dwarves 1/2] github CI: Build pahole "next" branch as well as latest changes

[Alan Maguire]

In order to support compare/contrast of BTF generation with
changes, build next branch of pahole for comparison purposes.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 .github/workflows/test.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index f11ebfe..99c179a 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -20,12 +20,12 @@ jobs:
             runs_on: 'ubuntu-24.04'
             arch: 'x86_64'
             llvm-version: '18'
-            pahole: 'master'
+            pahole: 'next'
           - kernel: 'LATEST'
             runs_on: 'ubuntu-24.04-arm'
             arch: 'aarch64'
             llvm-version: '18'
-            pahole: 'tmp.master'
+            pahole: 'next'
     name: Linux ${{ matrix.kernel }}
     uses: ./.github/workflows/vmtest.yml
     with:
-- 
2.43.5

Re: [PATCH v2 dwarves 1/2] github CI: Build pahole "next" branch as well as latest changes

[Ihor Solodrai]

On 6/30/25 3:15 AM, Alan Maguire wrote:
> In order to support compare/contrast of BTF generation with
> changes, build next branch of pahole for comparison purposes.
> 
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>   .github/workflows/test.yml | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)

Reviewed-by: Ihor Solodrai <ihor.solodrai@linux.dev>

> 
> diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
> index f11ebfe..99c179a 100644
> --- a/.github/workflows/test.yml
> +++ b/.github/workflows/test.yml
> @@ -20,12 +20,12 @@ jobs:
>               runs_on: 'ubuntu-24.04'
>               arch: 'x86_64'
>               llvm-version: '18'
> -            pahole: 'master'
> +            pahole: 'next'
>             - kernel: 'LATEST'
>               runs_on: 'ubuntu-24.04-arm'
>               arch: 'aarch64'
>               llvm-version: '18'
> -            pahole: 'tmp.master'
> +            pahole: 'next'
>       name: Linux ${{ matrix.kernel }}
>       uses: ./.github/workflows/vmtest.yml
>       with:

[PATCH v2 dwarves 2/2] github CI: Add comparison of generated BTF functions between baseline, change

[Alan Maguire]

Sometimes changes can be introduced that modify the set of functions
encoded in BTF, or change aspects of that encoding.  Add a non-fatal
comparison job to compare between the change and the base branch,
by default the "next" branch.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 .github/scripts/compare-functions.sh | 30 ++++++++++++++++++++++++++++
 .github/workflows/vmtest.yml         |  4 ++++
 2 files changed, 34 insertions(+)
 create mode 100755 .github/scripts/compare-functions.sh

diff --git a/.github/scripts/compare-functions.sh b/.github/scripts/compare-functions.sh
new file mode 100755
index 0000000..062f15c
--- /dev/null
+++ b/.github/scripts/compare-functions.sh
@@ -0,0 +1,30 @@
+#!/usr/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (c) 2025, Oracle and/or its affiliates.
+#
+
+GITHUB_WORKSPACE=${GITHUB_WORKSPACE:-$(pwd)}
+REPO_TARGET=${GITHUB_WORKSPACE}/.kernel
+VMLINUX=${GITHUB_WORKSPACE}/.kernel/vmlinux
+SELFTESTS=${GITHUB_WORKSPACE}/tests
+export PATH=${GITHUB_WORKSPACE}/install/usr/local/bin:${PATH}
+which pahole
+pahole --version
+cd $REPO_TARGET
+pfunct --all --format_path=btf $VMLINUX > functions_latest
+# now use baseline pahole for comparison
+export PAHOLE=/usr/local/bin/pahole
+rm -f vmlinux vmlinux.o
+export PATH=/usr/local/bin:${PATH}
+make oldconfig
+make -j $((4*$(nproc))) all
+pfunct --all --format_path=btf $VMLINUX > functions_base
+echo "Comparing vmlinux BTF functions generated with this change vs baseline."
+echo "Differences are non-fatal to the workflow, but should be examined for correctness."
+set +e
+diff functions_base functions_latest
+if [[ $? -eq 0 ]]; then
+	echo "Function lists are identical."
+fi
+set -e
diff --git a/.github/workflows/vmtest.yml b/.github/workflows/vmtest.yml
index 0f66eed..54bb92e 100644
--- a/.github/workflows/vmtest.yml
+++ b/.github/workflows/vmtest.yml
@@ -60,3 +60,7 @@ jobs:
         shell: bash
         run: .github/scripts/run-selftests.sh
 
+      - name: Compare functions generated
+        shell: bash
+        run: .github/scripts/compare-functions.sh
+
-- 
2.43.5

Re: [PATCH v2 dwarves 2/2] github CI: Add comparison of generated BTF functions between baseline, change

[Ihor Solodrai]

On 6/30/25 3:15 AM, Alan Maguire wrote:
> Sometimes changes can be introduced that modify the set of functions
> encoded in BTF, or change aspects of that encoding.  Add a non-fatal
> comparison job to compare between the change and the base branch,
> by default the "next" branch.
> 
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>   .github/scripts/compare-functions.sh | 30 ++++++++++++++++++++++++++++
>   .github/workflows/vmtest.yml         |  4 ++++
>   2 files changed, 34 insertions(+)
>   create mode 100755 .github/scripts/compare-functions.sh
> 

Hi Alan. That's a good addition. See a couple of comments below.

> diff --git a/.github/scripts/compare-functions.sh b/.github/scripts/compare-functions.sh
> new file mode 100755
> index 0000000..062f15c
> --- /dev/null
> +++ b/.github/scripts/compare-functions.sh
> @@ -0,0 +1,30 @@
> +#!/usr/bin/bash
> +# SPDX-License-Identifier: GPL-2.0-only
> +#
> +# Copyright (c) 2025, Oracle and/or its affiliates.
> +#
> +
> +GITHUB_WORKSPACE=${GITHUB_WORKSPACE:-$(pwd)}
> +REPO_TARGET=${GITHUB_WORKSPACE}/.kernel
> +VMLINUX=${GITHUB_WORKSPACE}/.kernel/vmlinux
> +SELFTESTS=${GITHUB_WORKSPACE}/tests

nit: SELFTESTS isn't used

> +export PATH=${GITHUB_WORKSPACE}/install/usr/local/bin:${PATH}
> +which pahole
> +pahole --version
> +cd $REPO_TARGET
> +pfunct --all --format_path=btf $VMLINUX > functions_latest
> +# now use baseline pahole for comparison
> +export PAHOLE=/usr/local/bin/pahole

So we assume that the baseline is installed at this path?
And that would be an installation done by
libbpf/ci/setup-build-env action?

I would not rely on that behavior.  I think a better approach is to
build and use master (or whatever is the baseline) explicitly.

> +rm -f vmlinux vmlinux.o
> +export PATH=/usr/local/bin:${PATH}
> +make oldconfig
> +make -j $((4*$(nproc))) all
> +pfunct --all --format_path=btf $VMLINUX > functions_base
> +echo "Comparing vmlinux BTF functions generated with this change vs baseline."
> +echo "Differences are non-fatal to the workflow, but should be examined for correctness."

You might find it useful to dump a formatted diff to
$GITHUB_STEP_SUMMARY, to get a rendered output in github UI.

See here: 
https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions?versionId=free-pro-team%40latest&productId=actions#adding-a-job-summary 


> +set +e
> +diff functions_base functions_latest
> +if [[ $? -eq 0 ]]; then
> +	echo "Function lists are identical."
> +fi
> +set -e
> diff --git a/.github/workflows/vmtest.yml b/.github/workflows/vmtest.yml
> index 0f66eed..54bb92e 100644
> --- a/.github/workflows/vmtest.yml
> +++ b/.github/workflows/vmtest.yml
> @@ -60,3 +60,7 @@ jobs:
>           shell: bash
>           run: .github/scripts/run-selftests.sh
>   
> +      - name: Compare functions generated
> +        shell: bash
> +        run: .github/scripts/compare-functions.sh
> +

Re: [PATCH v2 dwarves 2/2] github CI: Add comparison of generated BTF functions between baseline, change

[Alan Maguire]

On 30/06/2025 19:36, Ihor Solodrai wrote:
> On 6/30/25 3:15 AM, Alan Maguire wrote:
>> Sometimes changes can be introduced that modify the set of functions
>> encoded in BTF, or change aspects of that encoding.  Add a non-fatal
>> comparison job to compare between the change and the base branch,
>> by default the "next" branch.
>>
>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>> ---
>>   .github/scripts/compare-functions.sh | 30 ++++++++++++++++++++++++++++
>>   .github/workflows/vmtest.yml         |  4 ++++
>>   2 files changed, 34 insertions(+)
>>   create mode 100755 .github/scripts/compare-functions.sh
>>
> 
> Hi Alan. That's a good addition. See a couple of comments below.
> 

Thanks for taking a look! Replies below..

>> diff --git a/.github/scripts/compare-functions.sh b/.github/scripts/
>> compare-functions.sh
>> new file mode 100755
>> index 0000000..062f15c
>> --- /dev/null
>> +++ b/.github/scripts/compare-functions.sh
>> @@ -0,0 +1,30 @@
>> +#!/usr/bin/bash
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +#
>> +# Copyright (c) 2025, Oracle and/or its affiliates.
>> +#
>> +
>> +GITHUB_WORKSPACE=${GITHUB_WORKSPACE:-$(pwd)}
>> +REPO_TARGET=${GITHUB_WORKSPACE}/.kernel
>> +VMLINUX=${GITHUB_WORKSPACE}/.kernel/vmlinux
>> +SELFTESTS=${GITHUB_WORKSPACE}/tests
> 
> nit: SELFTESTS isn't used
>

good catch! will remove

>> +export PATH=${GITHUB_WORKSPACE}/install/usr/local/bin:${PATH}
>> +which pahole
>> +pahole --version
>> +cd $REPO_TARGET
>> +pfunct --all --format_path=btf $VMLINUX > functions_latest
>> +# now use baseline pahole for comparison
>> +export PAHOLE=/usr/local/bin/pahole
> 
> So we assume that the baseline is installed at this path?
> And that would be an installation done by
> libbpf/ci/setup-build-env action?
> 
> I would not rely on that behavior.  I think a better approach is to
> build and use master (or whatever is the baseline) explicitly.
> 

sure I can change this for v3; likely patch 1 will be changed then to
pahole: 'none' since there's no advantage in building pahole as part of
the setup-build-env step.

>> +rm -f vmlinux vmlinux.o
>> +export PATH=/usr/local/bin:${PATH}
>> +make oldconfig
>> +make -j $((4*$(nproc))) all
>> +pfunct --all --format_path=btf $VMLINUX > functions_base
>> +echo "Comparing vmlinux BTF functions generated with this change vs
>> baseline."
>> +echo "Differences are non-fatal to the workflow, but should be
>> examined for correctness."
> 
> You might find it useful to dump a formatted diff to
> $GITHUB_STEP_SUMMARY, to get a rendered output in github UI.
> 
> See here: https://docs.github.com/en/actions/reference/workflow-
> commands-for-github-actions?versionId=free-pro-
> team%40latest&productId=actions#adding-a-job-summary
>

great, thanks for the tip!

Alan