Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[Domenico Andreoli]

[-- Attachment #1: Type: text/plain, Size: 2753 bytes --]

Hi Arnaldo,

On Thu, Jul 18, 2024 at 11:56:09PM +0200, Ben Hutchings wrote:
> Package: pahole
> Version: 1.27-1
> Severity: normal
> X-Debbugs-Cc: debian-kernel@lists.debian.org
> 
> Several kernel builds on powerpc have failed recently:
> 
> 6.8.12-1:    https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.8.12-1&stamp=1717234422&raw=1
> 6.9.9-1:     https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.9.9-1&stamp=1720906547&raw=1
> 6.10-1~exp1: https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.10-1%7Eexp1&stamp=1721287862&raw=1
> 
> Note, these log files are up to 270 MB in size and should be
> downloaded; at least Firefox becomes unresponsive when trying to
> display them.
> 
> For each of these, the failure seems to start with an error from
> pahole such as:
> 
>     [102044] ARRAY (anon) type_id=99491 index_type_id=14 nr_elems=12 Error emitting BTF type
>     Encountered error while encoding BTF.

Does the above error ring any bell?

Is there anything I can do to help?

> 
> This does *not* happen consistently.  Compare these successful
> builds:
> 
> 6.8.12-1:    https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.8.12-1&stamp=1717278092&raw=1
> - This same version failed to build on the first try.
> 6.9.7-1:     https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.9.7-1&stamp=1719538806&raw=1
> - Earlier and later 6.9.x versions failed to build.
> 
> Both pahole versions 1.26-1 and 1.27-1 have been used in both
> successful and failing builds.
> 
> Ben.
> 
> -- System Information:
> Debian Release: trixie/sid
>   APT prefers unstable-debug
>   APT policy: (500, 'unstable-debug'), (500, 'stable-updates'), (500, 'stable-security'), (500, 'oldstable-updates'), (500, 'oldstable-security'), (500, 'oldoldstable-updates'), (500, 'oldoldstable'), (500, 'unstable'), (500, 'stable'), (500, 'oldstable'), (1, 'experimental')
> Architecture: amd64 (x86_64)
> Foreign Architectures: i386
> 
> Kernel: Linux 6.9.9-amd64 (SMP w/12 CPU threads; PREEMPT)
> Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE not set
> Shell: /bin/sh linked to /usr/bin/dash
> Init: systemd (via /run/systemd/system)
> LSM: AppArmor: enabled
> 
> Versions of packages pahole depends on:
> ii  libbpf1     1:1.4.3-1
> ii  libc6       2.39-4
> ii  libdw1t64   0.191-2
> ii  libelf1t64  0.191-2
> ii  zlib1g      1:1.3.dfsg+really1.3.1-1
> 
> pahole recommends no packages.
> 
> pahole suggests no packages.
> 
> -- debconf-show failed

-- 
rsa4096: 3B10 0CA1 8674 ACBA B4FE  FCD2 CE5B CF17 9960 DE13
ed25519: FFB4 0CC3 7F2E 091D F7DA  356E CC79 2832 ED38 CB05

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[Domenico Andreoli]

[-- Attachment #1: Type: text/plain, Size: 1734 bytes --]

On Fri, Jul 19, 2024 at 08:53:24AM +0200, Domenico Andreoli wrote:
> Hi Arnaldo,
> 
> On Thu, Jul 18, 2024 at 11:56:09PM +0200, Ben Hutchings wrote:
> > Package: pahole
> > Version: 1.27-1
> > Severity: normal
> > X-Debbugs-Cc: debian-kernel@lists.debian.org
> > 
> > Several kernel builds on powerpc have failed recently:
> > 
> > 6.8.12-1:    https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.8.12-1&stamp=1717234422&raw=1
> > 6.9.9-1:     https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.9.9-1&stamp=1720906547&raw=1
> > 6.10-1~exp1: https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.10-1%7Eexp1&stamp=1721287862&raw=1
> > 
> > Note, these log files are up to 270 MB in size and should be
> > downloaded; at least Firefox becomes unresponsive when trying to
> > display them.
> > 
> > For each of these, the failure seems to start with an error from
> > pahole such as:
> > 
> >     [102044] ARRAY (anon) type_id=99491 index_type_id=14 nr_elems=12 Error emitting BTF type
> >     Encountered error while encoding BTF.
> 
> Does the above error ring any bell?
> 
> Is there anything I can do to help?
> 

Additional info

Debian package 1.26-1 has no patches applied, 1.27-1 instead has the
following backports:

https://salsa.debian.org/debian/dwarves/-/blob/debian/1.27-1/debian/patches/00-core-Initialize-cu_node-with-INIT_LIST_HEAD.patch

https://salsa.debian.org/debian/dwarves/-/blob/debian/1.27-1/debian/patches/01-dwarf_loader-Add-missing-cus__add-to-cus__merge_and_process_cu.patch

Dom

-- 
rsa4096: 3B10 0CA1 8674 ACBA B4FE  FCD2 CE5B CF17 9960 DE13
ed25519: FFB4 0CC3 7F2E 091D F7DA  356E CC79 2832 ED38 CB05

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[Arnaldo Carvalho de Melo]

Adding Alan and Jiri to the CC list.

On Fri, Jul 19, 2024 at 08:53:24AM +0200, Domenico Andreoli wrote:
> Hi Arnaldo,
> 
> On Thu, Jul 18, 2024 at 11:56:09PM +0200, Ben Hutchings wrote:
> > Package: pahole
> > Version: 1.27-1
> > Severity: normal
> > X-Debbugs-Cc: debian-kernel@lists.debian.org
> > 
> > Several kernel builds on powerpc have failed recently:
> > 
> > 6.8.12-1:    https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.8.12-1&stamp=1717234422&raw=1
> > 6.9.9-1:     https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.9.9-1&stamp=1720906547&raw=1
> > 6.10-1~exp1: https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.10-1%7Eexp1&stamp=1721287862&raw=1
> > 
> > Note, these log files are up to 270 MB in size and should be
> > downloaded; at least Firefox becomes unresponsive when trying to
> > display them.
> > 
> > For each of these, the failure seems to start with an error from
> > pahole such as:
> > 
> >     [102044] ARRAY (anon) type_id=99491 index_type_id=14 nr_elems=12 Error emitting BTF type
> >     Encountered error while encoding BTF.
> 
> Does the above error ring any bell?

Nope
 
> Is there anything I can do to help?

From the 6.10-1~exp1:
https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.10-1%7Eexp1&stamp=1721287862&raw=1
file:

+ LLVM_OBJCOPY=powerpc-linux-gnu-objcopy pahole -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust .tmp_vmlinux.btf

Can I have access to that .tmp_vmlinux.btf file so that I can try to
reproduce it here?

- Arnaldo
 
> > 
> > This does *not* happen consistently.  Compare these successful
> > builds:
> > 
> > 6.8.12-1:    https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.8.12-1&stamp=1717278092&raw=1
> > - This same version failed to build on the first try.
> > 6.9.7-1:     https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.9.7-1&stamp=1719538806&raw=1
> > - Earlier and later 6.9.x versions failed to build.
> > 
> > Both pahole versions 1.26-1 and 1.27-1 have been used in both
> > successful and failing builds.
> > 
> > Ben.
> > 
> > -- System Information:
> > Debian Release: trixie/sid
> >   APT prefers unstable-debug
> >   APT policy: (500, 'unstable-debug'), (500, 'stable-updates'), (500, 'stable-security'), (500, 'oldstable-updates'), (500, 'oldstable-security'), (500, 'oldoldstable-updates'), (500, 'oldoldstable'), (500, 'unstable'), (500, 'stable'), (500, 'oldstable'), (1, 'experimental')
> > Architecture: amd64 (x86_64)
> > Foreign Architectures: i386
> > 
> > Kernel: Linux 6.9.9-amd64 (SMP w/12 CPU threads; PREEMPT)
> > Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE not set
> > Shell: /bin/sh linked to /usr/bin/dash
> > Init: systemd (via /run/systemd/system)
> > LSM: AppArmor: enabled
> > 
> > Versions of packages pahole depends on:
> > ii  libbpf1     1:1.4.3-1
> > ii  libc6       2.39-4
> > ii  libdw1t64   0.191-2
> > ii  libelf1t64  0.191-2
> > ii  zlib1g      1:1.3.dfsg+really1.3.1-1
> > 
> > pahole recommends no packages.
> > 
> > pahole suggests no packages.
> > 
> > -- debconf-show failed
> 
> -- 
> rsa4096: 3B10 0CA1 8674 ACBA B4FE  FCD2 CE5B CF17 9960 DE13
> ed25519: FFB4 0CC3 7F2E 091D F7DA  356E CC79 2832 ED38 CB05

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[Domenico Andreoli]

[-- Attachment #1: Type: text/plain, Size: 2167 bytes --]

CCing debian-kernel and debian-powerpc

On Fri, Jul 19, 2024 at 04:13:00PM -0300, Arnaldo Carvalho de Melo wrote:
> Adding Alan and Jiri to the CC list.
> 
> On Fri, Jul 19, 2024 at 08:53:24AM +0200, Domenico Andreoli wrote:
> > Hi Arnaldo,
> > 
> > On Thu, Jul 18, 2024 at 11:56:09PM +0200, Ben Hutchings wrote:
> > > Package: pahole
> > > Version: 1.27-1
> > > Severity: normal
> > > X-Debbugs-Cc: debian-kernel@lists.debian.org
> > > 
> > > Several kernel builds on powerpc have failed recently:
> > > 
> > > 6.8.12-1:    https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.8.12-1&stamp=1717234422&raw=1
> > > 6.9.9-1:     https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.9.9-1&stamp=1720906547&raw=1
> > > 6.10-1~exp1: https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.10-1%7Eexp1&stamp=1721287862&raw=1
> > > 
> > > Note, these log files are up to 270 MB in size and should be
> > > downloaded; at least Firefox becomes unresponsive when trying to
> > > display them.
> > > 
> > > For each of these, the failure seems to start with an error from
> > > pahole such as:
> > > 
> > >     [102044] ARRAY (anon) type_id=99491 index_type_id=14 nr_elems=12 Error emitting BTF type
> > >     Encountered error while encoding BTF.
> > 
> > Does the above error ring any bell?
> 
> Nope
>  
> > Is there anything I can do to help?
> 
> From the 6.10-1~exp1:
> https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.10-1%7Eexp1&stamp=1721287862&raw=1
> file:
> 
> + LLVM_OBJCOPY=powerpc-linux-gnu-objcopy pahole -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust .tmp_vmlinux.btf
> 
> Can I have access to that .tmp_vmlinux.btf file so that I can try to
> reproduce it here?

I don't have access to the build host (blaauw2) and I've some doubts
it would still have that file.

Maybe our kernel team and powerpc porters have suggestions?

Dom

-- 
rsa4096: 3B10 0CA1 8674 ACBA B4FE  FCD2 CE5B CF17 9960 DE13
ed25519: FFB4 0CC3 7F2E 091D F7DA  356E CC79 2832 ED38 CB05

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[John Paul Adrian Glaubitz]

Hi Domenico,

On Fri, 2024-07-19 at 23:20 +0200, Domenico Andreoli wrote:
> > > Is there anything I can do to help?
> > 
> > From the 6.10-1~exp1:
> > https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.10-1%7Eexp1&stamp=1721287862&raw=1
> > file:
> > 
> > + LLVM_OBJCOPY=powerpc-linux-gnu-objcopy pahole -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust .tmp_vmlinux.btf
> > 
> > Can I have access to that .tmp_vmlinux.btf file so that I can try to
> > reproduce it here?
> 
> I don't have access to the build host (blaauw2) and I've some doubts
> it would still have that file.
> 
> Maybe our kernel team and powerpc porters have suggestions?

I have root access to all powerpc/ppc64 machines (buildds and porterbox).

I'm cleaning up the porterbox now, disk is quite full, then you can try
to build the kernel package on perotto.debian.net or I can try it myself.

I have seen the bug myself and I wanted to debug it, but the attempt was
foiled by the fact that the disk on perotto is full (again).

Will take care of it and let you know when it's (some hours).

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[Domenico Andreoli]

[-- Attachment #1: Type: text/plain, Size: 1657 bytes --]

On Sat, Jul 20, 2024 at 09:16:24AM +0200, John Paul Adrian Glaubitz wrote:
> Hi Domenico,

Hi!

> On Fri, 2024-07-19 at 23:20 +0200, Domenico Andreoli wrote:
> > > > Is there anything I can do to help?
> > > 
> > > From the 6.10-1~exp1:
> > > https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.10-1%7Eexp1&stamp=1721287862&raw=1
> > > file:
> > > 
> > > + LLVM_OBJCOPY=powerpc-linux-gnu-objcopy pahole -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust .tmp_vmlinux.btf
> > > 
> > > Can I have access to that .tmp_vmlinux.btf file so that I can try to
> > > reproduce it here?
> > 
> > I don't have access to the build host (blaauw2) and I've some doubts
> > it would still have that file.
> > 
> > Maybe our kernel team and powerpc porters have suggestions?
> 
> I have root access to all powerpc/ppc64 machines (buildds and porterbox).
> 
> I'm cleaning up the porterbox now, disk is quite full, then you can try
> to build the kernel package on perotto.debian.net or I can try it myself.
> 
> I have seen the bug myself and I wanted to debug it, but the attempt was
> foiled by the fact that the disk on perotto is full (again).
> 
> Will take care of it and let you know when it's (some hours).

That's great, thank you.

> 
> Adrian
> 
> -- 
>  .''`.  John Paul Adrian Glaubitz
> : :' :  Debian Developer
> `. `'   Physicist
>   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

-- 
rsa4096: 3B10 0CA1 8674 ACBA B4FE  FCD2 CE5B CF17 9960 DE13
ed25519: FFB4 0CC3 7F2E 091D F7DA  356E CC79 2832 ED38 CB05

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 1653 bytes --]

On Sat, 2024-07-20 at 09:16 +0200, John Paul Adrian Glaubitz wrote:
> Hi Domenico,
> 
> On Fri, 2024-07-19 at 23:20 +0200, Domenico Andreoli wrote:
> > > > Is there anything I can do to help?
> > > 
> > > From the 6.10-1~exp1:
> > > https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.10-1%7Eexp1&stamp=1721287862&raw=1
> > > file:
> > > 
> > > + LLVM_OBJCOPY=powerpc-linux-gnu-objcopy pahole -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust .tmp_vmlinux.btf
> > > 
> > > Can I have access to that .tmp_vmlinux.btf file so that I can try to
> > > reproduce it here?
> > 
> > I don't have access to the build host (blaauw2) and I've some doubts
> > it would still have that file.
> > 
> > Maybe our kernel team and powerpc porters have suggestions?
> 
> I have root access to all powerpc/ppc64 machines (buildds and porterbox).
> 
> I'm cleaning up the porterbox now, disk is quite full, then you can try
> to build the kernel package on perotto.debian.net or I can try it myself.
> 
> I have seen the bug myself and I wanted to debug it, but the attempt was
> foiled by the fact that the disk on perotto is full (again).

I had a go yesterday and ran into the same problem.  I couldn't
reproduce with a small kernel config (allnoconfig + BPF + DEBUG_INFO +
DEBUG_INFO_BTF) and there wasn't enough disk space to build even one of
the Debian kernel flavours.

> Will take care of it and let you know when it's (some hours).

Thank you!

Ben.

-- 
Ben Hutchings
This sentence contradicts itself - no actually it doesn't.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[John Paul Adrian Glaubitz]

On Sat, 2024-07-20 at 21:17 +0200, Ben Hutchings wrote:
> I had a go yesterday and ran into the same problem.  I couldn't
> reproduce with a small kernel config (allnoconfig + BPF + DEBUG_INFO +
> DEBUG_INFO_BTF) and there wasn't enough disk space to build even one of
> the Debian kernel flavours.
> 
> > Will take care of it and let you know when it's (some hours).
> 
> Thank you!

There are now 120 GB of free disk space. Let me know if that's sufficient
or whether I need to clean up more, probably asking others to clean up
their home directories.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 2731 bytes --]

On Sat, 2024-07-20 at 21:45 +0200, John Paul Adrian Glaubitz wrote:
> On Sat, 2024-07-20 at 21:17 +0200, Ben Hutchings wrote:
> > I had a go yesterday and ran into the same problem.  I couldn't
> > reproduce with a small kernel config (allnoconfig + BPF + DEBUG_INFO +
> > DEBUG_INFO_BTF) and there wasn't enough disk space to build even one of
> > the Debian kernel flavours.
> > 
> > > Will take care of it and let you know when it's (some hours).
> > 
> > Thank you!
> 
> There are now 120 GB of free disk space. Let me know if that's sufficient
> or whether I need to clean up more, probably asking others to clean up
> their home directories.

I've now done 10 kernel builds on perotto (4 builds of just the
"powerpc" flavour and then 2 builds of all 3 flavours) and not
reproduced this.  I'm thinking this may be machine-dependent in some
way.

Looking again at all the build logs since DEBUG_INFO_BTF was enabled
for powerpc, we have:

Successes:

Version        Builder
----------------------------------
6.9.10-1       debian-project-be-2
6.9.7-1        debian-project-be-1
6.8.12-1       debian-project-be-2
6.8.11-1       blaauw
6.8.9-1        debian-project-be-2
6.7.12-1       blaauw
6.7.9-2        debian-project-be-2
6.7.4-1~exp1   blaauw
6.7.1-1~exp1   debian-project-be-2
6.6.15-1       blaauw
6.6.13-1       blaauw
6.6.11-1       debian-project-be-1
6.6.9-1+b1     debian-project-be-1
6.6.8-1        debian-project-be-2
6.6.4-1~exp1   blaauw
6.5.13-1       debian-project-be-1
6.5.10-1       debian-project-be-2
6.5.8-1        blaauw
6.5.6-1        debian-project-be-2
6.5.3-1        blaauw
6.5~rc7-1~exp1 debian-project-be-1
6.5~rc6-1~exp1 blaauw
6.4.13-1       blaauw
6.4.11-1       blaauw
6.4.4-3        blaauw

Failures:

Version        Builder             Failure mode
----------------------------------------------------------------------------
6.10-1~exp1    blaauw              this bug
6.9.9-1        blaauw              this bug
6.9.8-1        kapitsa             this bug
6.9.2-1~exp1   blaauw              this bug
6.8.12-1       kapitsa             this bug
6.7-1~exp1     debian-project-be-2 compiler OOM; not powerpc-specific
6.6.3-1~exp1   blaauw              kernel-wedge failed; not powerpc-specific
6.4.4-2        blaauw              out of disk space

Ignoring the unrelated failures, kapitsa has a 0% success rate (but
with only 2 attempts), blaauw an 80% success rate, and debian-project-
be-{1,2} have 100% success rates.

I don't know what differences there are between these builders that
might be relevant.

Ben.

-- 
Ben Hutchings
One of the nice things about standards is that
there are so many of them.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[John Paul Adrian Glaubitz]

Hi Ben,

On Mon, 2024-07-22 at 00:07 +0200, Ben Hutchings wrote:
> I don't know what differences there are between these builders that
> might be relevant.

For kapitsa, the installed host system is powerpc while all the others
run the ppc64 port.

As for the hardware:

kapitsa runs bare-metal (inside an LPAR) on a POWER8 machine:

root@kapitsa:~# grep model /proc/cpuinfo 
model           : IBM,8284-22A
root@kapitsa:~#

Both blaauw and perotto are KVMs running on watson which runs
Debian's ppc64el port (little-endian):

root@watson:~# grep model /proc/cpuinfo 
model           : 8247-42L
root@watson:~#

root@blaauw:~# grep model /proc/cpuinfo 
model           : IBM pSeries (emulated by qemu)
root@blaauw:~#

root@perotto:~# grep model /proc/cpuinfo 
model           : IBM pSeries (emulated by qemu)
root@perotto:~#

Both debian-project-be-01 and debian-project-be-02 are KVMs running
on OpenStack at OSUOSL's OpenPOWER platform:

root@debian-project-be-1:~# grep model /proc/cpuinfo 
model           : IBM pSeries (emulated by qemu)
root@debian-project-be-1:~#

root@debian-project-be-2:~# grep model /proc/cpuinfo 
model           : IBM pSeries (emulated by qemu)
root@debian-project-be-2:~#

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[Alan Maguire]

On 19/07/2024 20:13, Arnaldo Carvalho de Melo wrote:
> Adding Alan and Jiri to the CC list.
>

I'm late chiming in on this one, but judging by the output:

  BTF     .btf.vmlinux.bin.o
+ LLVM_OBJCOPY=powerpc-linux-gnu-objcopy pahole -J -j
--btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func
--lang_exclude=rust .tmp_vmlinux.btf
[102044] ARRAY (anon) type_id=99491 index_type_id=14 nr_elems=12 Error
emitting BTF type
Encountered error while encoding BTF.


...we hit an error in btf_encoder__add_array() as a result of
btf__add_array() failing:

btf__log_err(btf, BTF_KIND_ARRAY, NULL, true,
                              "type_id=%u index_type_id=%u nr_elems=%u
Error emitting BTF type",
                              type, index_type, nelems);


Unfortunately we don't preserve the negative id value (containing the
error code) in btf__log_err(); I'm thinking one thing we should do is
modify btf__log_err() to preserves errors for cases where the encoding
errors out due to a libbpf-returned -errno, something like


-__attribute ((format (printf, 5, 6)))
+__attribute ((format (printf, 6, 7)))
-static void btf__log_err(const struct btf *btf, int kind, const char *name,
+static void btf__log_err(const struct btf *btf, int libbpf_err, int
kind, const char *name,
                         bool output_cr, const char *fmt, ...)
{
        fprintf(stderr, "[%u] %s %s", btf__type_cnt(btf),
                btf_kind_str[kind], name ?: "(anon)");

	if (fmt && *fmt) {
                va_list ap;

                fprintf(stderr, " ");
                va_start(ap, fmt);
                vfprintf(stderr, fmt, ap);
                va_end(ap);
        }

+	if (libbpf_err)
+		fprintf(stderr, " libbpf error %d", libbpf_err);
        if (output_cr)
                fprintf(stderr, "\n");
}


So at least if this error recurs we'd have a clearer picture of what's
happening in libbpf. What do you think? I'll submit a patch for this if
it makes sense.

Alan

Re: Bug#1076564: pahole BTF processing seems flaky on powerpc

[Arnaldo Carvalho de Melo]

On Wed, Jul 24, 2024 at 11:04:51AM +0100, Alan Maguire wrote:
> On 19/07/2024 20:13, Arnaldo Carvalho de Melo wrote:
> > Adding Alan and Jiri to the CC list.
> >
> 
> I'm late chiming in on this one, but judging by the output:
> 
>   BTF     .btf.vmlinux.bin.o
> + LLVM_OBJCOPY=powerpc-linux-gnu-objcopy pahole -J -j
> --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func
> --lang_exclude=rust .tmp_vmlinux.btf
> [102044] ARRAY (anon) type_id=99491 index_type_id=14 nr_elems=12 Error
> emitting BTF type
> Encountered error while encoding BTF.
> 
> 
> ...we hit an error in btf_encoder__add_array() as a result of
> btf__add_array() failing:
> 
> btf__log_err(btf, BTF_KIND_ARRAY, NULL, true,
>                               "type_id=%u index_type_id=%u nr_elems=%u
> Error emitting BTF type",
>                               type, index_type, nelems);
> 
> 
> Unfortunately we don't preserve the negative id value (containing the
> error code) in btf__log_err(); I'm thinking one thing we should do is
> modify btf__log_err() to preserves errors for cases where the encoding
> errors out due to a libbpf-returned -errno, something like
> 
> 
> -__attribute ((format (printf, 5, 6)))
> +__attribute ((format (printf, 6, 7)))
> -static void btf__log_err(const struct btf *btf, int kind, const char *name,
> +static void btf__log_err(const struct btf *btf, int libbpf_err, int
> kind, const char *name,
>                          bool output_cr, const char *fmt, ...)
> {
>         fprintf(stderr, "[%u] %s %s", btf__type_cnt(btf),
>                 btf_kind_str[kind], name ?: "(anon)");
> 
> 	if (fmt && *fmt) {
>                 va_list ap;
> 
>                 fprintf(stderr, " ");
>                 va_start(ap, fmt);
>                 vfprintf(stderr, fmt, ap);
>                 va_end(ap);
>         }
> 
> +	if (libbpf_err)
> +		fprintf(stderr, " libbpf error %d", libbpf_err);
>         if (output_cr)
>                 fprintf(stderr, "\n");
> }
> 
> 
> So at least if this error recurs we'd have a clearer picture of what's
> happening in libbpf. What do you think? I'll submit a patch for this if
> it makes sense.

I agree completely that the error reporting we have is lacking, we
better go and add extra info for these cases so that we can more quickly
get a clue of what is taking place, so please submit patches for that
and I'll consider them.

Thanks,

- Arnaldo