Dwarves debugging tools
 help / color / mirror / Atom feed
From: Alan Maguire <alan.maguire@oracle.com>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Andrii Nakryiko <andrii@kernel.org>, Eduard <eddyz87@gmail.com>,
	Ihor Solodrai <ihor.solodrai@linux.dev>,
	bpf <bpf@vger.kernel.org>,
	dwarves@vger.kernel.org,
	Kumar Kartikeya Dwivedi <memxor@gmail.com>
Subject: Re: pahole and gcc-14 issues
Date: Fri, 25 Apr 2025 18:50:24 +0100	[thread overview]
Message-ID: <076e52f6-248a-4a41-a199-3c705cb3d3c5@oracle.com> (raw)
In-Reply-To: <CAADnVQL+-LiJGXwxD3jEUrOonO-fX0SZC8496dVzUXvfkB7gYQ@mail.gmail.com>

On 25/04/2025 15:50, Alexei Starovoitov wrote:
> Hi All,
> 
> Looks like pahole fails to deduplicate BTF when kernel and
> kernel module are built with gcc-14.
> I see this issue with various kernel .config-s on bpf and
> bpf-next trees.
> I tried pahole 1.28 and the latest master. Same issues.
> 
> BTF in bpf_testmod.ko built with gcc-14 has 2849 types.
> When built with gcc-13 it has 454 types.
> So something is confusing dedup logic.
> Would be great if dedup experts can take a look,
> since this dedup issue is breaking a lot of selftests/bpf.
> 
> Also vmlinux.h generated out of the kernel compiled with gcc-13
> and out of the kernel compiled with gcc-14 shows these differences:
> 
> --- vmlinux13.h    2025-04-24 21:33:50.556884372 -0700
> +++ vmlinux14.h    2025-04-24 21:39:10.310488992 -0700
> @@ -148815,7 +148815,6 @@
>  extern int hid_bpf_input_report(struct hid_bpf_ctx *ctx, enum
> hid_report_type type, u8 *buf, const size_t buf__sz) __weak __ksym;
>  extern void hid_bpf_release_context(struct hid_bpf_ctx *ctx) __weak __ksym;
>  extern int hid_bpf_try_input_report(struct hid_bpf_ctx *ctx, enum
> hid_report_type type, u8 *buf, const size_t buf__sz) __weak __ksym;
> -extern bool scx_bpf_consume(u64 dsq_id) __weak __ksym;
>  extern int scx_bpf_cpu_node(s32 cpu) __weak __ksym;
>  extern struct rq *scx_bpf_cpu_rq(s32 cpu) __weak __ksym;
>  extern u32 scx_bpf_cpuperf_cap(s32 cpu) __weak __ksym;
> @@ -148825,12 +148824,8 @@
>  extern void scx_bpf_destroy_dsq(u64 dsq_id) __weak __ksym;
>  extern void scx_bpf_dispatch(struct task_struct *p, u64 dsq_id, u64
> slice, u64 enq_flags) __weak __ksym;
>  extern void scx_bpf_dispatch_cancel(void) __weak __ksym;
> -extern bool scx_bpf_dispatch_from_dsq(struct bpf_iter_scx_dsq
> *it__iter, struct task_struct *p, u64 dsq_id, u64 enq_flags) __weak
> __ksym;
> -extern void scx_bpf_dispatch_from_dsq_set_slice(struct
> bpf_iter_scx_dsq *it__iter, u64 slice) __weak __ksym;
>  extern void scx_bpf_dispatch_from_dsq_set_vtime(struct
> bpf_iter_scx_dsq *it__iter, u64 vtime) __weak __ksym;
>  extern u32 scx_bpf_dispatch_nr_slots(void) __weak __ksym;
> -extern void scx_bpf_dispatch_vtime(struct task_struct *p, u64 dsq_id,
> u64 slice, u64 vtime, u64 enq_flags) __weak __ksym;
> -extern bool scx_bpf_dispatch_vtime_from_dsq(struct bpf_iter_scx_dsq
> *it__iter, struct task_struct *p, u64 dsq_id, u64 enq_flags) __weak
> __ksym;
>  extern void scx_bpf_dsq_insert(struct task_struct *p, u64 dsq_id, u64
> slice, u64 enq_flags) __weak __ksym;
>  extern void scx_bpf_dsq_insert_vtime(struct task_struct *p, u64
> dsq_id, u64 slice, u64 vtime, u64 enq_flags) __weak __ksym;
>  extern bool scx_bpf_dsq_move(struct bpf_iter_scx_dsq *it__iter,
> struct task_struct *p, u64 dsq_id, u64 enq_flags) __weak __ksym;
> 
> gcc-14's kernel is clearly wrong.
> These 5 kfuncs still exist in the kernel.
> I manually checked there is no if __GNUC__ > 13 in the code.
> Also:
> nm bld/vmlinux|grep -w scx_bpf_consume
> ffffffff8159d4b0 T scx_bpf_consume
> ffffffff8120ea81 t scx_bpf_consume.cold
> 
> I suspect the second issue is not related to the dedup problem.
> All 5 missing kfuncs have ".cold" optimized bodies.
> But ".cold" maybe a red herring, since
> nm bld/vmlinux|grep -w scx_bpf_dispatch
> ffffffff8159d020 T scx_bpf_dispatch
> ffffffff8120ea0f t scx_bpf_dispatch.cold
> but this kfunc is present in vmlinux14.h
> 
> If it makes a difference I have these configs:
> # CONFIG_DEBUG_INFO_DWARF4 is not set
> # CONFIG_DEBUG_INFO_DWARF5 is not set
> # CONFIG_DEBUG_INFO_REDUCED is not set
> CONFIG_DEBUG_INFO_COMPRESSED_NONE=y
> # CONFIG_DEBUG_INFO_COMPRESSED_ZLIB is not set
> # CONFIG_DEBUG_INFO_SPLIT is not set
> CONFIG_DEBUG_INFO_BTF=y
> CONFIG_PAHOLE_HAS_SPLIT_BTF=y
> CONFIG_DEBUG_INFO_BTF_MODULES=y

thanks for the report! I've just reproduced this now with gcc 14; my
initial theory was it might be DWARF5-related, but dedup issues occur
for modules with CONFIG_DEBUG_INFO_DWARF4=y also. I'm seeing task_struct
duplicates in module BTF among other things, so I will try and dig
further and report back when I find something. Like you I suspect the
issues with missing kfuncs are different; may be an issue with our logic
handling inconsistent functions getting confused by the .cold
components. But right now understanding dedup issues is the top priority.

Alan

  reply	other threads:[~2025-04-25 17:50 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-04-25 14:50 pahole and gcc-14 issues Alexei Starovoitov
2025-04-25 17:50 ` Alan Maguire [this message]
2025-04-25 17:58   ` Andrii Nakryiko
2025-04-25 20:36     ` Alan Maguire
2025-04-25 20:41       ` Andrii Nakryiko
2025-04-26 17:28         ` Alan Maguire
2025-04-28 15:21           ` Alan Maguire
2025-04-28 19:36             ` Arnaldo Carvalho de Melo
2025-04-28 19:50               ` Arnaldo Carvalho de Melo
2025-04-28 22:12             ` Alexei Starovoitov
2025-04-29  0:33               ` Andrii Nakryiko
2025-04-29  6:59                 ` Andrii Nakryiko
2025-04-29 15:37                   ` Andrii Nakryiko
2025-04-29 15:55                     ` Alan Maguire
2025-04-29 19:29                     ` Eduard Zingerman
2025-04-29 19:50                       ` Andrii Nakryiko
2025-04-29 20:34                         ` Eduard Zingerman
2025-04-29 20:55                         ` Alexei Starovoitov
2025-04-29 21:44                           ` Andrii Nakryiko
2025-04-25 20:58     ` Andrii Nakryiko
2025-04-25 18:14 ` Eduard Zingerman
2025-04-25 19:43   ` Eduard Zingerman
2025-04-25 20:16     ` Alexei Starovoitov
2025-04-25 20:25       ` Eduard Zingerman
2025-04-25 20:30         ` Alexei Starovoitov
2025-04-25 20:30         ` Eduard Zingerman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=076e52f6-248a-4a41-a199-3c705cb3d3c5@oracle.com \
    --to=alan.maguire@oracle.com \
    --cc=acme@kernel.org \
    --cc=alexei.starovoitov@gmail.com \
    --cc=andrii@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=dwarves@vger.kernel.org \
    --cc=eddyz87@gmail.com \
    --cc=ihor.solodrai@linux.dev \
    --cc=memxor@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox