Re: [PATCH bpf v2 2/2] bpf: Harden __bpf_kfunc tag against linker kfunc removal

Jiri Olsa
2024-06-26T09:52:09Z

On Tue, Jun 25, 2024 at 12:46:48PM +0200, Geert Uytterhoeven wrote:
> Hi Tony,
>
> On Mon, 3 Jun 2024, Tony Ambardar wrote:
> > BPF kfuncs are often not directly referenced and may be inadvertently
> > removed by optimization steps during kernel builds, thus the __bpf_kfunc
> > tag mitigates against this removal by including the __used macro. However,
> > this macro alone does not prevent removal during linking, and may still
> > yield build warnings (e.g. on mips64el):
> >
> > LD vmlinux
> > BTFIDS vmlinux
> > WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature
> > WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key
> > WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key
> > WARN: resolve_btfids: unresolved symbol bpf_key_put
> > WARN: resolve_btfids: unresolved symbol bpf_iter_task_next
> > WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new
> > WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr
> > WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry
> > WARN: resolve_btfids: unresolved symbol bpf_cgroup_release
> > WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id
> > WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire
> > WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages
> > NM System.map
> > SORTTAB vmlinux
> > OBJCOPY vmlinux.32
> >
> > Update the __bpf_kfunc tag to better guard against linker optimization by
> > including the new __retain compiler macro, which fixes the warnings above.
> >
> > Verify the __retain macro with readelf by checking object flags for 'R':
> >
> > $ readelf -Wa kernel/trace/bpf_trace.o
> > Section Headers:
> > [Nr] Name Type Address Off Size ES Flg Lk Inf Al
> > ...
> > [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8
> > ...
> > Key to Flags:
> > ...
> > R (retain), D (mbind), p (processor specific)
> >
> > Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/
> > Reported-by: kernel test robot <lkp@intel.com>
> > Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/
> > Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs")
> > Cc: stable@vger.kernel.org # v6.6+
> > Signed-off-by: Tony Ambardar <Tony.Ambardar@gmail.com>
>
> Thanks for your patch, which is now commit 7bdcedd5c8fb88e7
> ("bpf: Harden __bpf_kfunc tag against linker kfunc removal") in
> v6.10-rc5.
>
> This is causing build failures on ARM with
> CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y:
>
> net/core/filter.c:11859:1: error: ‘retain’ attribute ignored [-Werror=attributes]
> 11859 | {
> | ^
> net/core/filter.c:11872:1: error: ‘retain’ attribute ignored [-Werror=attributes]
> 11872 | {
> | ^
> net/core/filter.c:11885:1: error: ‘retain’ attribute ignored [-Werror=attributes]
> 11885 | {
> | ^
> net/core/filter.c:11906:1: error: ‘retain’ attribute ignored [-Werror=attributes]
> 11906 | {
> | ^
> net/core/filter.c:12092:1: error: ‘retain’ attribute ignored [-Werror=attributes]
> 12092 | {
> | ^
> net/core/xdp.c:713:1: error: ‘retain’ attribute ignored [-Werror=attributes]
> 713 | {
> | ^
> net/core/xdp.c:736:1: error: ‘retain’ attribute ignored [-Werror=attributes]
> 736 | {
> | ^
> net/core/xdp.c:769:1: error: ‘retain’ attribute ignored [-Werror=attributes]
> 769 | {
> | ^
> [...]
>
> My compiler is arm-linux-gnueabihf-gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04).

hum, so it'd mean __has_attribute(__retain__) returns true while gcc still
ignores the retain attribute.. like in this bug which seems similar:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99587
but not sure how it got fixed.. any chance you can upgrade gcc and retest?

jirka

>
> > --- a/include/linux/btf.h
> > +++ b/include/linux/btf.h
> > @@ -82,7 +82,7 @@
> > * as to avoid issues such as the compiler inlining or eliding either a static
> > * kfunc, or a global kfunc in an LTO build.
> > */
> > -#define __bpf_kfunc __used noinline
> > +#define __bpf_kfunc __used __retain noinline
> >
> > #define __bpf_kfunc_start_defs() \
> > __diag_push(); \
>
> Gr{oetje,eeting}s,
>
> Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
> -- Linus Torvalds