Re: Question: fentry on kernel func optimized by compiler

[Tao Chen <chen.dylane@linux.dev>]

在 2025/4/16 03:21, Jiri Olsa 写道:
> On Tue, Apr 15, 2025 at 08:10:09PM +0800, Tao Chen wrote:
>> 在 2025/3/31 18:13, Alan Maguire 写道:
>>> On 28/03/2025 17:21, Andrii Nakryiko wrote:
>>>> On Thu, Mar 27, 2025 at 9:03 AM Tao Chen <chen.dylane@linux.dev> wrote:
>>>>>
>>>>> Hi,
> 
> hi, sorry for late reply
> 
>>>>>
>>>>> I recently encountered a problem when using fentry to trace kernel
>>>>> functions optimized by compiler, the specific situation is as follows:
>>>>> https://github.com/bpftrace/bpftrace/issues/3940
>>>>>
>>>>> Simply put, some functions have been optimized by the compiler. The
>>>>> original function names are found through BTF, but the optimized
>>>>> functions are the ones that exist in kallsyms_lookup_name. Therefore,
>>>>> the two do not match.
> 
> hum, would you have example BTF/kernel/config with such case? I'd think
> if function is in BTF you should find it through kallsyms, the other way
> around is not garanteed
> 

It seems that there is no special configuration. I am using the official 
Ubuntu 24.04. The issue link is: 
https://github.com/bpftrace/bpftrace/issues/3940

>>>>>
>>>>>            func_proto = btf_type_by_id(desc_btf, func->type);
>>>>>            if (!func_proto || !btf_type_is_func_proto(func_proto)) {
>>>>>                    verbose(env, "kernel function btf_id %u does not have a
>>>>> valid func_proto\n",
>>>>>                            func_id);
>>>>>                    return -EINVAL;
>>>>>            }
>>>>>
>>>>>            func_name = btf_name_by_offset(desc_btf, func->name_off);
>>>>>            addr = kallsyms_lookup_name(func_name);
>>>>>            if (!addr) {
>>>>>                    verbose(env, "cannot find address for kernel function
>>>>> %s\n",
>>>>>                            func_name);
>>>>>                    return -EINVAL;
>>>>>            }
>>>>>
>>>>> I have made a simple statistics and there are approximately more than
>>>>> 2,000 functions in Ubuntu 24.04.
>>>>>
>>>>> dylane@2404:~$ cat /proc/kallsyms | grep isra | wc -l
>>>>> 2324
>>>>>
>>>>> So can we add a judgment from libbpf. If it is an optimized function,
>>>>
>>>> No, we cannot. It's a different function at that point and libbpf
>>>> isn't going to be in the business of guessing on behalf of the user
>>>> whether it's ok to do or not.
>>>>
>>>> But the user can use multi-kprobe with `prefix*` naming, if they
>>>> encountered (or are anticipating) this situation and think it's fine
>>>> for them.
>>>>
>>>> As for fentry/fexit, you need to have the correct BTF ID associated
>>>> with that function anyways, so I'm not sure that currently you can
>>>> attach fentry/fexit to such compiler-optimized functions at all
>>>> (pahole won't produce BTF for such functions, right?).
>>>>
>>>
>>> Yep, BTF will not be there for all cases, but ever since we've had the
>>> "optimized_func" BTF feature, we've have encoded BTF for suffixed
>>> functions as long as their parameters are not optimized away and as long
>>> as we don't have multiple inconsistent representations associated with a
>>> function (say two differing function signatures for the same name).
>>> Optimization away of parameters happens quite frequently, but not always
>>> for .isra.0 functions so they are potentially sometimes safe for fentry.
>>>
>>> The complication here is that - by design - the function name in BTF
>>> will be the prefix; i.e. "foo" not "foo.isra.0". So how we match up the
>>> BTF with the right suffixed function is an issue; a single function
>>> prefix can have ".isra.0" and ".cold.0" suffixes associated for example.
>>> The latter isn't really a function entry point (in the C code at least);
>>> it's just a split of the function into common path and less common path
>>> for better code locality for the more commonly-executed code.
>>>
>>> Yonghong and I talked about this a bit last year in Plumbers, but it did
>>> occur to me that there are conditions where we could match up the prefix
>>> from BTF with a guaranteed fentry point for the function using info we
>>> have today.
>>>
>>> /sys/kernel/tracing/available_filter_functions_addr has similar info to
>>> /proc/kallysyms but as far as I understand it we are also guaranteed
>>> that the associated addresses correspond to real function entry points.
> 
> available_filter_functions_addr contains exactly the same functions as
> available_filter_functions (all functions managed by ftrace), it just
> adds address value for each function as a hint to user space
> 
> it's used in kprobe_multi bench test to get all traceable addresses,
> and passed to kprobe_multi link, which uses following fprobe interface
> to attach:
> 
>          err = register_fprobe_ips(&link->fp, addrs, cnt);
> 
>>> So because the BTF representation currently ensures consistency _and_
>>> available function parameters, I think we could use
>>> available_filter_functions_addr to carry out the match and provide the
>>> right function address for the BTF representation.
> 
> where would you do the match to available_filter_functions_addr?

maybe in libbpf? If it is the situation encountered above, use the 
passed-in address from user; otherwise, still use the original logic.

	func_name = btf_name_by_offset(desc_btf, func->name_off);
	addr = kallsyms_lookup_name(func_name);
> 
> also not sure how it's connected to the original issue, that seems
> to be related to pahole eliminating unsafe functions in BTF?
> 
> jirka
> 
> 
>>>
>>
>> Hi, Alan
>> Sorry for not replying in time. As you said，it seems much simpler when use
>> the func addr from available_filter_functions_addr. It seems to be a bit
>> similar to the way of passing function addresses in kprobe_multi. @Andrii
>> @Jiri what do you think?
>>
>>> In the future, the hope is we can handle inconsistent representations
>>> too in BTF, but the above represents a possible approach we could
>>> implement today I think, though I may be missing something. Thanks!
>>>
>>> Alan
>>
>>
>> -- 
>> Best Regards
>> Tao Chen
>>


-- 
Best Regards
Tao Chen