[PATCH] bpf: Mark kfuncs as __noclone

[PATCH] bpf: Mark kfuncs as __noclone

[Andrea Righi]

Some distributions (e.g., CachyOS) support building the kernel with -O3,
but doing so may break kfuncs, resulting in their symbols not being
properly exported.

In fact, with gcc -O3, some kfuncs may be optimized away despite being
annotated as noinline. This happens because gcc can still clone the
function during IPA optimizations, e.g., by duplicating or inlining it
into callers, and then dropping the standalone symbol. This breaks BTF
ID resolution since resolve_btfids relies on the presence of a global
symbol for each kfunc.

Currently, this is not an issue for upstream, because we don't allow
building the kernel with -O3, but it may be safer to address it anyway,
to prevent potential issues in the future if compilers become more
aggressive with optimizations.

Therefore, add __noclone to __bpf_kfunc to ensure kfuncs are never
cloned and remain distinct, globally visible symbols, regardless of
the optimization level.

Fixes: 57e7c169cd6af ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs")
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 include/linux/btf.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/btf.h b/include/linux/btf.h
index 9eda6b113f9b4..f06976ffb63f9 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -86,7 +86,7 @@
  * as to avoid issues such as the compiler inlining or eliding either a static
  * kfunc, or a global kfunc in an LTO build.
  */
-#define __bpf_kfunc __used __retain noinline
+#define __bpf_kfunc __used __retain __noclone noinline
 
 #define __bpf_kfunc_start_defs()					       \
 	__diag_push();							       \
-- 
2.50.1

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Yonghong Song]

On 8/22/25 7:05 AM, Andrea Righi wrote:
> Some distributions (e.g., CachyOS) support building the kernel with -O3,
> but doing so may break kfuncs, resulting in their symbols not being
> properly exported.
>
> In fact, with gcc -O3, some kfuncs may be optimized away despite being
> annotated as noinline. This happens because gcc can still clone the
> function during IPA optimizations, e.g., by duplicating or inlining it
> into callers, and then dropping the standalone symbol. This breaks BTF
> ID resolution since resolve_btfids relies on the presence of a global
> symbol for each kfunc.
>
> Currently, this is not an issue for upstream, because we don't allow
> building the kernel with -O3, but it may be safer to address it anyway,
> to prevent potential issues in the future if compilers become more
> aggressive with optimizations.
>
> Therefore, add __noclone to __bpf_kfunc to ensure kfuncs are never
> cloned and remain distinct, globally visible symbols, regardless of
> the optimization level.

I tried with gcc14 and can reproduced the issue described in the above.
I build the kernel like below with gcc14
   make KCFLAGS='-O3' -j
and get the following build error
   WARN: resolve_btfids: unresolved symbol bpf_strnchr
   make[2]: *** [/home/yhs/work/bpf-next/scripts/Makefile.vmlinux:91: vmlinux] Error 255
   make[2]: *** Deleting file 'vmlinux'
Checking the symbol table:
    22276: ffffffff81b15260   249 FUNC    LOCAL  DEFAULT    1 bpf_strnchr.cons[...]
   235128: ffffffff81b1f540   296 FUNC    GLOBAL DEFAULT    1 bpf_strnchr
and the disasm code:
   bpf_strnchr:
     ...

   bpf_strchr:
     ...
     bpf_strnchr.constprop.0
     ...

So in symbol table, we have both bpf_strnchr.constprop.0 and bpf_strnchr.
For such case, pahole will skip func bpf_strnchr hence the above resolve_btfids
failure.

The solution in this patch can indeed resolve this issue.

>
> Fixes: 57e7c169cd6af ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs")
> Signed-off-by: Andrea Righi <arighi@nvidia.com>
> ---
>   include/linux/btf.h | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/include/linux/btf.h b/include/linux/btf.h
> index 9eda6b113f9b4..f06976ffb63f9 100644
> --- a/include/linux/btf.h
> +++ b/include/linux/btf.h
> @@ -86,7 +86,7 @@
>    * as to avoid issues such as the compiler inlining or eliding either a static
>    * kfunc, or a global kfunc in an LTO build.
>    */
> -#define __bpf_kfunc __used __retain noinline
> +#define __bpf_kfunc __used __retain __noclone noinline
>   
>   #define __bpf_kfunc_start_defs()					       \
>   	__diag_push();							       \

The llvm does not support __noclone so __noclone is noop for llvm.
I tried with
   make KCFLAGS='-O3' LLVM=1 -j
and the kernel is build successfully and also the kernel can boot properly.

So ack the patch:
Acked-by: Yonghong Song <yonghong.song@linux.dev>

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Eduard Zingerman]

On Tue, 2025-08-26 at 13:17 -0700, Yonghong Song wrote:

[...]

> I tried with gcc14 and can reproduced the issue described in the above.
> I build the kernel like below with gcc14
>    make KCFLAGS='-O3' -j
> and get the following build error
>    WARN: resolve_btfids: unresolved symbol bpf_strnchr
>    make[2]: *** [/home/yhs/work/bpf-next/scripts/Makefile.vmlinux:91: vmlinux] Error 255
>    make[2]: *** Deleting file 'vmlinux'
> Checking the symbol table:
>     22276: ffffffff81b15260   249 FUNC    LOCAL  DEFAULT    1 bpf_strnchr.cons[...]
>    235128: ffffffff81b1f540   296 FUNC    GLOBAL DEFAULT    1 bpf_strnchr
> and the disasm code:
>    bpf_strnchr:
>      ...
> 
>    bpf_strchr:
>      ...
>      bpf_strnchr.constprop.0
>      ...
> 
> So in symbol table, we have both bpf_strnchr.constprop.0 and bpf_strnchr.
> For such case, pahole will skip func bpf_strnchr hence the above resolve_btfids
> failure.
> 
> The solution in this patch can indeed resolve this issue.

It looks like instead of adding __noclone there is an option to
improve pahole's filtering of ambiguous functions.
Abstractly, there is nothing wrong with having a clone of a global
function that has undergone additional optimizations. As long as the
original symbol exists, everything should be fine.

Since kfuncs are global, this should guarantee that the compiler does not
change their signature, correct? Does this also hold for LTO builds?
If so, when pahole sees a set of symbols like [foo, foo.1, foo.2, ...],
with 'foo' being global and the rest local, then there is no real need
to filter out 'foo'.

Wdyt?

[...]

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Andrea Righi]

On Tue, Aug 26, 2025 at 10:02:31PM -0700, Eduard Zingerman wrote:
> On Tue, 2025-08-26 at 13:17 -0700, Yonghong Song wrote:
> 
> [...]
> 
> > I tried with gcc14 and can reproduced the issue described in the above.
> > I build the kernel like below with gcc14
> >    make KCFLAGS='-O3' -j
> > and get the following build error
> >    WARN: resolve_btfids: unresolved symbol bpf_strnchr
> >    make[2]: *** [/home/yhs/work/bpf-next/scripts/Makefile.vmlinux:91: vmlinux] Error 255
> >    make[2]: *** Deleting file 'vmlinux'
> > Checking the symbol table:
> >     22276: ffffffff81b15260   249 FUNC    LOCAL  DEFAULT    1 bpf_strnchr.cons[...]
> >    235128: ffffffff81b1f540   296 FUNC    GLOBAL DEFAULT    1 bpf_strnchr
> > and the disasm code:
> >    bpf_strnchr:
> >      ...
> > 
> >    bpf_strchr:
> >      ...
> >      bpf_strnchr.constprop.0
> >      ...
> > 
> > So in symbol table, we have both bpf_strnchr.constprop.0 and bpf_strnchr.
> > For such case, pahole will skip func bpf_strnchr hence the above resolve_btfids
> > failure.
> > 
> > The solution in this patch can indeed resolve this issue.
> 
> It looks like instead of adding __noclone there is an option to
> improve pahole's filtering of ambiguous functions.
> Abstractly, there is nothing wrong with having a clone of a global
> function that has undergone additional optimizations. As long as the
> original symbol exists, everything should be fine.
> 
> Since kfuncs are global, this should guarantee that the compiler does not
> change their signature, correct? Does this also hold for LTO builds?
> If so, when pahole sees a set of symbols like [foo, foo.1, foo.2, ...],
> with 'foo' being global and the rest local, then there is no real need
> to filter out 'foo'.
> 
> Wdyt?

I think we should do both: fix resolve_btfids to ignore compiler
optimization suffixes (.isra., .constprop., .part., .cold, ...) and add
__noclone.

This feels like the safest path IMHO. Fixing resolve_btfids alone works
with current compilers, but future compiler versions, under aggressive
IPA/LTO optimizations, might decide that the main global symbol is
redundant and drop it altogether, leading to similar issues.

Basically, fixing the tool makes the BTF pipeline more robust, adding
__noclone also makes the exported symbols themselves more robust,
regardless of compiler optimizations.

Thanks,
-Andrea

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Eduard Zingerman]

On Wed, 2025-08-27 at 07:41 +0200, Andrea Righi wrote:
> On Tue, Aug 26, 2025 at 10:02:31PM -0700, Eduard Zingerman wrote:
> > On Tue, 2025-08-26 at 13:17 -0700, Yonghong Song wrote:
> >
> > [...]
> >
> > > I tried with gcc14 and can reproduced the issue described in the above.
> > > I build the kernel like below with gcc14
> > >    make KCFLAGS='-O3' -j
> > > and get the following build error
> > >    WARN: resolve_btfids: unresolved symbol bpf_strnchr
> > >    make[2]: *** [/home/yhs/work/bpf-next/scripts/Makefile.vmlinux:91: vmlinux] Error 255
> > >    make[2]: *** Deleting file 'vmlinux'
> > > Checking the symbol table:
> > >     22276: ffffffff81b15260   249 FUNC    LOCAL  DEFAULT    1 bpf_strnchr.cons[...]
> > >    235128: ffffffff81b1f540   296 FUNC    GLOBAL DEFAULT    1 bpf_strnchr
> > > and the disasm code:
> > >    bpf_strnchr:
> > >      ...
> > >
> > >    bpf_strchr:
> > >      ...
> > >      bpf_strnchr.constprop.0
> > >      ...
> > >
> > > So in symbol table, we have both bpf_strnchr.constprop.0 and bpf_strnchr.
> > > For such case, pahole will skip func bpf_strnchr hence the above resolve_btfids
> > > failure.
> > >
> > > The solution in this patch can indeed resolve this issue.
> >
> > It looks like instead of adding __noclone there is an option to
> > improve pahole's filtering of ambiguous functions.
> > Abstractly, there is nothing wrong with having a clone of a global
> > function that has undergone additional optimizations. As long as the
> > original symbol exists, everything should be fine.
> >
> > Since kfuncs are global, this should guarantee that the compiler does not
> > change their signature, correct? Does this also hold for LTO builds?
> > If so, when pahole sees a set of symbols like [foo, foo.1, foo.2, ...],
> > with 'foo' being global and the rest local, then there is no real need
> > to filter out 'foo'.
> >
> > Wdyt?
>
> I think we should do both: fix resolve_btfids to ignore compiler
> optimization suffixes (.isra., .constprop., .part., .cold, ...) and add
> __noclone.
>
> This feels like the safest path IMHO. Fixing resolve_btfids alone works
> with current compilers, but future compiler versions, under aggressive
> IPA/LTO optimizations, might decide that the main global symbol is
> redundant and drop it altogether, leading to similar issues.
>
> Basically, fixing the tool makes the BTF pipeline more robust, adding
> __noclone also makes the exported symbols themselves more robust,
> regardless of compiler optimizations.

If we are being really paranoid about LTO builds, is __noclone sufficient?
E.g. [1] does not imply that signature can't be changed.
We currently apply only __retain__, here is a little test with both attributes:

    $ cat foo.c
    __attribute__((__noclone__, __retain__))
    int foo(int a) {
      return a;
    }

    $ cat main.c
    int foo(int);

    int main(int argc, char **argv) {
      return foo(0);
    }

    $ gcc -O3 -Wall -flto foo.c main.c -o a.out
    $ nm a.out | grep foo
    $ objdump -Sdr a.out | grep foo
    $ objdump -Sdr a.out | less
    $ nm a.out | grep foo | wc -l
    0
    $ objdump -Sdr a.out | grep foo | wc -l
    0

export.h:EXPORT_SYMBOL does the following trick:

  extern typeof(cachemode2protval) cachemode2protval;
  static void * __attribute__((__used__))
         __attribute__((__section__(".discard.addressable")))
         __UNIQUE_ID___addressable_cachemode2protval489 = (void *)(uintptr_t)&cachemode2protval;
         asm(".section \".export_symbol\",\"a\" ;\
              __export_symbol_cachemode2protval: ;\
              .asciz \"\" ;\
              .ascii \"\" \"\\0\" ;\
              .balign 8 ;\
              .quad cachemode2protval ;\
              .previous");

Should we employ something similar?

[1] https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-noclone-function-attribute

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Eduard Zingerman]

On Tue, 2025-08-26 at 23:52 -0700, Eduard Zingerman wrote:

[...]

> If we are being really paranoid about LTO builds, is __noclone sufficient?
> E.g. [1] does not imply that signature can't be changed.
> We currently apply only __retain__, here is a little test with both attributes:

Nope, there are also 'used' and 'noinline' applied.
With these the function is preserved as expected.
Sorry for the noise.

[...]

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Andrea Righi]

On Wed, Aug 27, 2025 at 12:01:09AM -0700, Eduard Zingerman wrote:
> On Tue, 2025-08-26 at 23:52 -0700, Eduard Zingerman wrote:
> 
> [...]
> 
> > If we are being really paranoid about LTO builds, is __noclone sufficient?
> > E.g. [1] does not imply that signature can't be changed.
> > We currently apply only __retain__, here is a little test with both attributes:
> 
> Nope, there are also 'used' and 'noinline' applied.
> With these the function is preserved as expected.
> Sorry for the noise.

Yeah, 'used' forces the function to be emitted even if it appears
unreferenced. Together with 'noclone', 'retain' and 'noinline', should
ensure the symbol exists and can be reliably found by resolve_btfids.

We could be extra paranoid and mimic EXPORT_SYMBOL(), moving the symbols to
their own section, but in practice I don't think this is strictly
necessary, even in presence of aggressive compiler / LTO optimizations.

Thanks,
-Andrea

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Yonghong Song]

On 8/26/25 11:52 PM, Eduard Zingerman wrote:
> On Wed, 2025-08-27 at 07:41 +0200, Andrea Righi wrote:
>> On Tue, Aug 26, 2025 at 10:02:31PM -0700, Eduard Zingerman wrote:
>>> On Tue, 2025-08-26 at 13:17 -0700, Yonghong Song wrote:
>>>
>>> [...]
>>>
>>>> I tried with gcc14 and can reproduced the issue described in the above.
>>>> I build the kernel like below with gcc14
>>>>     make KCFLAGS='-O3' -j
>>>> and get the following build error
>>>>     WARN: resolve_btfids: unresolved symbol bpf_strnchr
>>>>     make[2]: *** [/home/yhs/work/bpf-next/scripts/Makefile.vmlinux:91: vmlinux] Error 255
>>>>     make[2]: *** Deleting file 'vmlinux'
>>>> Checking the symbol table:
>>>>      22276: ffffffff81b15260   249 FUNC    LOCAL  DEFAULT    1 bpf_strnchr.cons[...]
>>>>     235128: ffffffff81b1f540   296 FUNC    GLOBAL DEFAULT    1 bpf_strnchr
>>>> and the disasm code:
>>>>     bpf_strnchr:
>>>>       ...
>>>>
>>>>     bpf_strchr:
>>>>       ...
>>>>       bpf_strnchr.constprop.0
>>>>       ...
>>>>
>>>> So in symbol table, we have both bpf_strnchr.constprop.0 and bpf_strnchr.
>>>> For such case, pahole will skip func bpf_strnchr hence the above resolve_btfids
>>>> failure.
>>>>
>>>> The solution in this patch can indeed resolve this issue.
>>> It looks like instead of adding __noclone there is an option to
>>> improve pahole's filtering of ambiguous functions.
>>> Abstractly, there is nothing wrong with having a clone of a global
>>> function that has undergone additional optimizations. As long as the
>>> original symbol exists, everything should be fine.
>>>
>>> Since kfuncs are global, this should guarantee that the compiler does not
>>> change their signature, correct? Does this also hold for LTO builds?
>>> If so, when pahole sees a set of symbols like [foo, foo.1, foo.2, ...],
>>> with 'foo' being global and the rest local, then there is no real need
>>> to filter out 'foo'.
>>>
>>> Wdyt?
>> I think we should do both: fix resolve_btfids to ignore compiler
>> optimization suffixes (.isra., .constprop., .part., .cold, ...) and add
>> __noclone.
>>
>> This feels like the safest path IMHO. Fixing resolve_btfids alone works
>> with current compilers, but future compiler versions, under aggressive
>> IPA/LTO optimizations, might decide that the main global symbol is
>> redundant and drop it altogether, leading to similar issues.
>>
>> Basically, fixing the tool makes the BTF pipeline more robust, adding
>> __noclone also makes the exported symbols themselves more robust,
>> regardless of compiler optimizations.
> If we are being really paranoid about LTO builds, is __noclone sufficient?
> E.g. [1] does not imply that signature can't be changed.
> We currently apply only __retain__, here is a little test with both attributes:
>
>      $ cat foo.c
>      __attribute__((__noclone__, __retain__))
>      int foo(int a) {
>        return a;
>      }
>
>      $ cat main.c
>      int foo(int);
>
>      int main(int argc, char **argv) {
>        return foo(0);
>      }
>
>      $ gcc -O3 -Wall -flto foo.c main.c -o a.out

Currently kernel does not support LTO for gcc build.

>      $ nm a.out | grep foo
>      $ objdump -Sdr a.out | grep foo
>      $ objdump -Sdr a.out | less
>      $ nm a.out | grep foo | wc -l
>      0
>      $ objdump -Sdr a.out | grep foo | wc -l
>      0
>
> export.h:EXPORT_SYMBOL does the following trick:
>
>    extern typeof(cachemode2protval) cachemode2protval;
>    static void * __attribute__((__used__))
>           __attribute__((__section__(".discard.addressable")))
>           __UNIQUE_ID___addressable_cachemode2protval489 = (void *)(uintptr_t)&cachemode2protval;
>           asm(".section \".export_symbol\",\"a\" ;\
>                __export_symbol_cachemode2protval: ;\
>                .asciz \"\" ;\
>                .ascii \"\" \"\\0\" ;\
>                .balign 8 ;\
>                .quad cachemode2protval ;\
>                .previous");
>
> Should we employ something similar?
>
> [1] https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-noclone-function-attribute

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Yonghong Song]

On 8/26/25 10:02 PM, Eduard Zingerman wrote:
> On Tue, 2025-08-26 at 13:17 -0700, Yonghong Song wrote:
>
> [...]
>
>> I tried with gcc14 and can reproduced the issue described in the above.
>> I build the kernel like below with gcc14
>>     make KCFLAGS='-O3' -j
>> and get the following build error
>>     WARN: resolve_btfids: unresolved symbol bpf_strnchr
>>     make[2]: *** [/home/yhs/work/bpf-next/scripts/Makefile.vmlinux:91: vmlinux] Error 255
>>     make[2]: *** Deleting file 'vmlinux'
>> Checking the symbol table:
>>      22276: ffffffff81b15260   249 FUNC    LOCAL  DEFAULT    1 bpf_strnchr.cons[...]
>>     235128: ffffffff81b1f540   296 FUNC    GLOBAL DEFAULT    1 bpf_strnchr
>> and the disasm code:
>>     bpf_strnchr:
>>       ...
>>
>>     bpf_strchr:
>>       ...
>>       bpf_strnchr.constprop.0
>>       ...
>>
>> So in symbol table, we have both bpf_strnchr.constprop.0 and bpf_strnchr.
>> For such case, pahole will skip func bpf_strnchr hence the above resolve_btfids
>> failure.
>>
>> The solution in this patch can indeed resolve this issue.
> It looks like instead of adding __noclone there is an option to
> improve pahole's filtering of ambiguous functions.
> Abstractly, there is nothing wrong with having a clone of a global
> function that has undergone additional optimizations. As long as the
> original symbol exists, everything should be fine.

Right. The generated code itself is totally fine. The problem is
currently pahole will filter out bpf_strnchr since in the symbol table
having both bpf_strnchr and bpf_strnchr.constprop.0. It there is
no explicit dwarf-level signature in dwarf for bpf_strnchr.constprop.0.
(For this particular .constprop.0 case, it is possible to derive the
  signature. but it will be hard for other suffixes like .isra).
The current pahole will have strip out suffixes so the function
name is 'bpf_strnchr' which covers bpf_strnchr and bpf_strnchr.constprop.0.
Since two underlying signature is different, the 'bpf_strnchr'
will be filtered out.

I am actually working to improve such cases in llvm to address
like foo() and foo.<...>() functions and they will have their
own respective functions. We will discuss with gcc folks
about how to implement similar approaches in gcc.


>
> Since kfuncs are global, this should guarantee that the compiler does not
> change their signature, correct? Does this also hold for LTO builds?

Yes, the original signature will not changed. This holds for LTO build
and global variables/functions will not be renamed.

> If so, when pahole sees a set of symbols like [foo, foo.1, foo.2, ...],

The compiler needs to emit the signature in dwarf for foo.1, foo.2, etc. and this
is something I am working on.

> with 'foo' being global and the rest local, then there is no real need
> to filter out 'foo'.

I think the current __noclone approach is okay as the full implementation
for signature changes (foo, foo.1, ...) might takes a while for both llvm
and gcc.

>
> Wdyt?
>
> [...]

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Eduard Zingerman]

On Wed, 2025-08-27 at 10:00 -0700, Yonghong Song wrote:
> 
> On 8/26/25 10:02 PM, Eduard Zingerman wrote:
> > On Tue, 2025-08-26 at 13:17 -0700, Yonghong Song wrote:
> > 
> > [...]
> > 
> > > I tried with gcc14 and can reproduced the issue described in the above.
> > > I build the kernel like below with gcc14
> > >     make KCFLAGS='-O3' -j
> > > and get the following build error
> > >     WARN: resolve_btfids: unresolved symbol bpf_strnchr
> > >     make[2]: *** [/home/yhs/work/bpf-next/scripts/Makefile.vmlinux:91: vmlinux] Error 255
> > >     make[2]: *** Deleting file 'vmlinux'
> > > Checking the symbol table:
> > >      22276: ffffffff81b15260   249 FUNC    LOCAL  DEFAULT    1 bpf_strnchr.cons[...]
> > >     235128: ffffffff81b1f540   296 FUNC    GLOBAL DEFAULT    1 bpf_strnchr
> > > and the disasm code:
> > >     bpf_strnchr:
> > >       ...
> > > 
> > >     bpf_strchr:
> > >       ...
> > >       bpf_strnchr.constprop.0
> > >       ...
> > > 
> > > So in symbol table, we have both bpf_strnchr.constprop.0 and bpf_strnchr.
> > > For such case, pahole will skip func bpf_strnchr hence the above resolve_btfids
> > > failure.
> > > 
> > > The solution in this patch can indeed resolve this issue.
> > It looks like instead of adding __noclone there is an option to
> > improve pahole's filtering of ambiguous functions.
> > Abstractly, there is nothing wrong with having a clone of a global
> > function that has undergone additional optimizations. As long as the
> > original symbol exists, everything should be fine.
> 
> Right. The generated code itself is totally fine. The problem is
> currently pahole will filter out bpf_strnchr since in the symbol table
> having both bpf_strnchr and bpf_strnchr.constprop.0. It there is
> no explicit dwarf-level signature in dwarf for bpf_strnchr.constprop.0.
> (For this particular .constprop.0 case, it is possible to derive the
>   signature. but it will be hard for other suffixes like .isra).
> The current pahole will have strip out suffixes so the function
> name is 'bpf_strnchr' which covers bpf_strnchr and bpf_strnchr.constprop.0.
> Since two underlying signature is different, the 'bpf_strnchr'
> will be filtered out.

Yes, I understand the mechanics. My question is: is it really
necessary for pahole to go through this process?

It sees two functions: 'bpf_strnchr', 'bpf_strnchr.constprop.0',
first global, second local, first with DWARF signature, second w/o
DWARF signature. So, why conflating the two?

For non-lto build the function being global guarantees signature
correctness, and below you confirm that it is the case for lto builds
as well. So, it looks like we are just loosing 'bpf_strnchr' for no
good reason.

> I am actually working to improve such cases in llvm to address
> like foo() and foo.<...>() functions and they will have their
> own respective functions. We will discuss with gcc folks
> about how to implement similar approaches in gcc.
> 
> > 
> > Since kfuncs are global, this should guarantee that the compiler does not
> > change their signature, correct? Does this also hold for LTO builds?
> 
> Yes, the original signature will not changed. This holds for LTO build
> and global variables/functions will not be renamed.
> 
> > If so, when pahole sees a set of symbols like [foo, foo.1, foo.2, ...],
> 
> The compiler needs to emit the signature in dwarf for foo.1, foo.2, etc. and this
> is something I am working on.
> 
> > with 'foo' being global and the rest local, then there is no real need
> > to filter out 'foo'.
> 
> I think the current __noclone approach is okay as the full implementation
> for signature changes (foo, foo.1, ...) might takes a while for both llvm
> and gcc.
> 
> > 
> > Wdyt?
> > 
> > [...]

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Alan Maguire]

On 27/08/2025 20:13, Eduard Zingerman wrote:
> On Wed, 2025-08-27 at 10:00 -0700, Yonghong Song wrote:
>>
>> On 8/26/25 10:02 PM, Eduard Zingerman wrote:
>>> On Tue, 2025-08-26 at 13:17 -0700, Yonghong Song wrote:
>>>
>>> [...]
>>>
>>>> I tried with gcc14 and can reproduced the issue described in the above.
>>>> I build the kernel like below with gcc14
>>>>     make KCFLAGS='-O3' -j
>>>> and get the following build error
>>>>     WARN: resolve_btfids: unresolved symbol bpf_strnchr
>>>>     make[2]: *** [/home/yhs/work/bpf-next/scripts/Makefile.vmlinux:91: vmlinux] Error 255
>>>>     make[2]: *** Deleting file 'vmlinux'
>>>> Checking the symbol table:
>>>>      22276: ffffffff81b15260   249 FUNC    LOCAL  DEFAULT    1 bpf_strnchr.cons[...]
>>>>     235128: ffffffff81b1f540   296 FUNC    GLOBAL DEFAULT    1 bpf_strnchr
>>>> and the disasm code:
>>>>     bpf_strnchr:
>>>>       ...
>>>>
>>>>     bpf_strchr:
>>>>       ...
>>>>       bpf_strnchr.constprop.0
>>>>       ...
>>>>
>>>> So in symbol table, we have both bpf_strnchr.constprop.0 and bpf_strnchr.
>>>> For such case, pahole will skip func bpf_strnchr hence the above resolve_btfids
>>>> failure.
>>>>
>>>> The solution in this patch can indeed resolve this issue.
>>> It looks like instead of adding __noclone there is an option to
>>> improve pahole's filtering of ambiguous functions.
>>> Abstractly, there is nothing wrong with having a clone of a global
>>> function that has undergone additional optimizations. As long as the
>>> original symbol exists, everything should be fine.
>>
>> Right. The generated code itself is totally fine. The problem is
>> currently pahole will filter out bpf_strnchr since in the symbol table
>> having both bpf_strnchr and bpf_strnchr.constprop.0. It there is
>> no explicit dwarf-level signature in dwarf for bpf_strnchr.constprop.0.
>> (For this particular .constprop.0 case, it is possible to derive the
>>   signature. but it will be hard for other suffixes like .isra).
>> The current pahole will have strip out suffixes so the function
>> name is 'bpf_strnchr' which covers bpf_strnchr and bpf_strnchr.constprop.0.
>> Since two underlying signature is different, the 'bpf_strnchr'
>> will be filtered out.
> 
> Yes, I understand the mechanics. My question is: is it really
> necessary for pahole to go through this process?
> 
> It sees two functions: 'bpf_strnchr', 'bpf_strnchr.constprop.0',
> first global, second local, first with DWARF signature, second w/o
> DWARF signature. So, why conflating the two?
> 
> For non-lto build the function being global guarantees signature
> correctness, and below you confirm that it is the case for lto builds
> as well. So, it looks like we are just loosing 'bpf_strnchr' for no
> good reason.
>

I'm working on a small 2-patch series at the moment to improve this. The
problem is we currently have no way to associate the DWARF with the
relevant ELF function; DWARF representations of functions do not have
"." suffixes either so we are just matching by name prefix when we
collect DWARF info about a particular function.

The series I'm working on uses DWARF addresses to improve the DWARF/ELF
association, ensuring that we don't toss functions that look
inconsistent but just have .part or .cold suffixed components that have
non-matching DWARF function signatures. ".constprop" isn't covered yet
however.

>> I am actually working to improve such cases in llvm to address
>> like foo() and foo.<...>() functions and they will have their
>> own respective functions. We will discuss with gcc folks
>> about how to implement similar approaches in gcc.
>>
>>>
>>> Since kfuncs are global, this should guarantee that the compiler does not
>>> change their signature, correct? Does this also hold for LTO builds?
>>
>> Yes, the original signature will not changed. This holds for LTO build
>> and global variables/functions will not be renamed.
>>
>>> If so, when pahole sees a set of symbols like [foo, foo.1, foo.2, ...],
>>
>> The compiler needs to emit the signature in dwarf for foo.1, foo.2, etc. and this
>> is something I am working on.
>>
>>> with 'foo' being global and the rest local, then there is no real need
>>> to filter out 'foo'.
>>
>> I think the current __noclone approach is okay as the full implementation
>> for signature changes (foo, foo.1, ...) might takes a while for both llvm
>> and gcc.
>>
>>>
>>> Wdyt?
>>>
>>> [...]

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Eduard Zingerman]

On Wed, 2025-08-27 at 20:28 +0100, Alan Maguire wrote:

[...]

> I'm working on a small 2-patch series at the moment to improve this. The
> problem is we currently have no way to associate the DWARF with the
> relevant ELF function; DWARF representations of functions do not have
> "." suffixes either so we are just matching by name prefix when we
> collect DWARF info about a particular function.

Oh, I see, there is no way to associate DWARF info with either
'bpf_strnchr' or 'bpf_strnchr.constprop.0' w/o checking address.
Thank you.

> The series I'm working on uses DWARF addresses to improve the DWARF/ELF
> association, ensuring that we don't toss functions that look
> inconsistent but just have .part or .cold suffixed components that have
> non-matching DWARF function signatures. ".constprop" isn't covered yet
> however.

Is ".constprop" special, or just has to be allowed as one of the prefixes?

[...]

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Alan Maguire]

On 27/08/2025 20:41, Eduard Zingerman wrote:
> On Wed, 2025-08-27 at 20:28 +0100, Alan Maguire wrote:
> 
> [...]
> 
>> I'm working on a small 2-patch series at the moment to improve this. The
>> problem is we currently have no way to associate the DWARF with the
>> relevant ELF function; DWARF representations of functions do not have
>> "." suffixes either so we are just matching by name prefix when we
>> collect DWARF info about a particular function.
> 
> Oh, I see, there is no way to associate DWARF info with either
> 'bpf_strnchr' or 'bpf_strnchr.constprop.0' w/o checking address.
> Thank you.
> 
>> The series I'm working on uses DWARF addresses to improve the DWARF/ELF
>> association, ensuring that we don't toss functions that look
>> inconsistent but just have .part or .cold suffixed components that have
>> non-matching DWARF function signatures. ".constprop" isn't covered yet
>> however.
> 
> Is ".constprop" special, or just has to be allowed as one of the prefixes?
>

Yonghong can remind me if I've got this wrong, but .constprop is
somewhat different from .part/.cold in that the latter aren't really on
function boundaries. Sometimes we want to retain .constprop
representations since they are function boundaries and sometimes do not
mess with parameters in incompatible ways. If we can find a good
heuristic for tossing them when they are not helpful as in the above
case that would be great, but I'm not sure how to do that without losing
BTF representations which are useful. Any suggestions on that would be
really great; in the meantime I'll try and get the series dealing with
.part and .cold functions out ASAP. Thanks!

Alan

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Yonghong Song]

On 8/27/25 12:52 PM, Alan Maguire wrote:
> On 27/08/2025 20:41, Eduard Zingerman wrote:
>> On Wed, 2025-08-27 at 20:28 +0100, Alan Maguire wrote:
>>
>> [...]
>>
>>> I'm working on a small 2-patch series at the moment to improve this. The
>>> problem is we currently have no way to associate the DWARF with the
>>> relevant ELF function; DWARF representations of functions do not have
>>> "." suffixes either so we are just matching by name prefix when we
>>> collect DWARF info about a particular function.
>> Oh, I see, there is no way to associate DWARF info with either
>> 'bpf_strnchr' or 'bpf_strnchr.constprop.0' w/o checking address.
>> Thank you.
>>
>>> The series I'm working on uses DWARF addresses to improve the DWARF/ELF
>>> association, ensuring that we don't toss functions that look
>>> inconsistent but just have .part or .cold suffixed components that have
>>> non-matching DWARF function signatures. ".constprop" isn't covered yet
>>> however.
>> Is ".constprop" special, or just has to be allowed as one of the prefixes?
>>
> Yonghong can remind me if I've got this wrong, but .constprop is
> somewhat different from .part/.cold in that the latter aren't really on

For symbol with .cold, it is not a function. It is just a jump target
from another function.

For symbol with .part, it is a actual function, but mostly like its
function signature has changed as it is part of the original
function.

For symbol with .constprop, is a clone of another function but
with less parameters, i.e., some parameters become a constant
inside the .constprop.<n> function.

With gcc build, you can see even more complicated suffixes:
   ffffffff81825bf0 t __remove_instance.part.0.constprop.0
   ffffffff81ed07c0 t eventfd_ctx_fileget.part.0.isra.0
   ...

> function boundaries. Sometimes we want to retain .constprop
> representations since they are function boundaries and sometimes do not
> mess with parameters in incompatible ways. If we can find a good
> heuristic for tossing them when they are not helpful as in the above
> case that would be great, but I'm not sure how to do that without losing

It is indeed very hard to have a good heuristic for those function
with suffixes. '<func>.constprop.<n>' might be easier as you can
check location in the subprogrm, if there is no location, most
likely that parameter has become a constant inside the function.

Currently I am working on llvm to add
   - function with suffixes
   - function with changed signature and without suffixes.

Such infomation should have better mapping from func to
its type.

> BTF representations which are useful. Any suggestions on that would be
> really great; in the meantime I'll try and get the series dealing with
> .part and .cold functions out ASAP. Thanks!
>
> Alan

Re: [PATCH] bpf: Mark kfuncs as __noclone

[Yonghong Song]

On 8/27/25 12:13 PM, Eduard Zingerman wrote:
> On Wed, 2025-08-27 at 10:00 -0700, Yonghong Song wrote:
>> On 8/26/25 10:02 PM, Eduard Zingerman wrote:
>>> On Tue, 2025-08-26 at 13:17 -0700, Yonghong Song wrote:
>>>
>>> [...]
>>>
>>>> I tried with gcc14 and can reproduced the issue described in the above.
>>>> I build the kernel like below with gcc14
>>>>      make KCFLAGS='-O3' -j
>>>> and get the following build error
>>>>      WARN: resolve_btfids: unresolved symbol bpf_strnchr
>>>>      make[2]: *** [/home/yhs/work/bpf-next/scripts/Makefile.vmlinux:91: vmlinux] Error 255
>>>>      make[2]: *** Deleting file 'vmlinux'
>>>> Checking the symbol table:
>>>>       22276: ffffffff81b15260   249 FUNC    LOCAL  DEFAULT    1 bpf_strnchr.cons[...]
>>>>      235128: ffffffff81b1f540   296 FUNC    GLOBAL DEFAULT    1 bpf_strnchr
>>>> and the disasm code:
>>>>      bpf_strnchr:
>>>>        ...
>>>>
>>>>      bpf_strchr:
>>>>        ...
>>>>        bpf_strnchr.constprop.0
>>>>        ...
>>>>
>>>> So in symbol table, we have both bpf_strnchr.constprop.0 and bpf_strnchr.
>>>> For such case, pahole will skip func bpf_strnchr hence the above resolve_btfids
>>>> failure.
>>>>
>>>> The solution in this patch can indeed resolve this issue.
>>> It looks like instead of adding __noclone there is an option to
>>> improve pahole's filtering of ambiguous functions.
>>> Abstractly, there is nothing wrong with having a clone of a global
>>> function that has undergone additional optimizations. As long as the
>>> original symbol exists, everything should be fine.
>> Right. The generated code itself is totally fine. The problem is
>> currently pahole will filter out bpf_strnchr since in the symbol table
>> having both bpf_strnchr and bpf_strnchr.constprop.0. It there is
>> no explicit dwarf-level signature in dwarf for bpf_strnchr.constprop.0.
>> (For this particular .constprop.0 case, it is possible to derive the
>>    signature. but it will be hard for other suffixes like .isra).
>> The current pahole will have strip out suffixes so the function
>> name is 'bpf_strnchr' which covers bpf_strnchr and bpf_strnchr.constprop.0.
>> Since two underlying signature is different, the 'bpf_strnchr'
>> will be filtered out.
> Yes, I understand the mechanics. My question is: is it really
> necessary for pahole to go through this process?
>
> It sees two functions: 'bpf_strnchr', 'bpf_strnchr.constprop.0',
> first global, second local, first with DWARF signature, second w/o
> DWARF signature. So, why conflating the two?

In this particular case, I think what you describe the correct.
For *Global* symbol 'bpf_strnchr', the signature should be in
the dwarf. But for *Local* symbol 'bpf_strnchr.constprop.0', the
signature is not clear. I suspect that pahole may not
distinguish between *Global* and *Local* symbols where they have
the same prefix.

The case like this patch to have a clone for a kfunc global
func should be very rare. That is another reason I think
__noclone should be good enough and it can reduce the
complexity in pahole. But I will be okay as well if the
consensus is to implement the support in pahole.

>
> For non-lto build the function being global guarantees signature
> correctness, and below you confirm that it is the case for lto builds
> as well. So, it looks like we are just loosing 'bpf_strnchr' for no
> good reason.
>
>> I am actually working to improve such cases in llvm to address
>> like foo() and foo.<...>() functions and they will have their
>> own respective functions. We will discuss with gcc folks
>> about how to implement similar approaches in gcc.
>>
>>> Since kfuncs are global, this should guarantee that the compiler does not
>>> change their signature, correct? Does this also hold for LTO builds?
>> Yes, the original signature will not changed. This holds for LTO build
>> and global variables/functions will not be renamed.
>>
>>> If so, when pahole sees a set of symbols like [foo, foo.1, foo.2, ...],
>> The compiler needs to emit the signature in dwarf for foo.1, foo.2, etc. and this
>> is something I am working on.
>>
>>> with 'foo' being global and the rest local, then there is no real need
>>> to filter out 'foo'.
>> I think the current __noclone approach is okay as the full implementation
>> for signature changes (foo, foo.1, ...) might takes a while for both llvm
>> and gcc.
>>
>>> Wdyt?
>>>
>>> [...]

Re: [PATCH] bpf: Mark kfuncs as __noclone

[David Vernet]

[-- Attachment #1: Type: text/plain, Size: 1249 bytes --]

On Fri, Aug 22, 2025 at 04:05:53PM +0200, Andrea Righi wrote:
> Some distributions (e.g., CachyOS) support building the kernel with -O3,
> but doing so may break kfuncs, resulting in their symbols not being
> properly exported.
> 
> In fact, with gcc -O3, some kfuncs may be optimized away despite being
> annotated as noinline. This happens because gcc can still clone the
> function during IPA optimizations, e.g., by duplicating or inlining it
> into callers, and then dropping the standalone symbol. This breaks BTF
> ID resolution since resolve_btfids relies on the presence of a global
> symbol for each kfunc.
> 
> Currently, this is not an issue for upstream, because we don't allow
> building the kernel with -O3, but it may be safer to address it anyway,
> to prevent potential issues in the future if compilers become more
> aggressive with optimizations.
> 
> Therefore, add __noclone to __bpf_kfunc to ensure kfuncs are never
> cloned and remain distinct, globally visible symbols, regardless of
> the optimization level.
> 
> Fixes: 57e7c169cd6af ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs")
> Signed-off-by: Andrea Righi <arighi@nvidia.com>

Acked-by: David Vernet <void@manifault.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]