Re: [PATCH] libbpf: kprobe.multi: Filter with blacklist and available_filter_functions

[Jackie Liu <liu.yun@linux.dev>]

Hi Jiri.

在 2023/5/24 09:03, Jackie Liu 写道:
> Hi Jiri.
> 
> 在 2023/5/24 00:17, Jiri Olsa 写道:
>> On Tue, May 23, 2023 at 09:25:47PM +0800, Jackie Liu wrote:
>>> From: Jackie Liu <liuyun01@kylinos.cn>
>>>
>>> When using regular expression matching with "kprobe multi", it scans all
>>> the functions under "/proc/kallsyms" that can be matched. However, 
>>> not all
>>> of them can be traced by kprobe.multi. If any one of the functions fails
>>> to be traced, it will result in the failure of all functions. The best
>>> approach is to filter out the functions that cannot be traced to ensure
>>> proper tracking of the functions.
>>>
>>> But, the addition of these checks will frequently probe whether a 
>>> function
>>> complies with "available_filter_functions" and ensure that it has not 
>>> been
>>> filtered by kprobe's blacklist. As a result, it may take a longer time
>>> during startup. The function implementation is referenced from BCC's
>>> "kprobe_exists()"
>>>
>>> Here is the test eBPF program [1].
>>> [1] 
>>> https://github.com/JackieLiu1/ketones/commit/a9e76d1ba57390e533b8b3eadde97f7a4535e867
>>>
>>> Signed-off-by: Jackie Liu <liuyun01@kylinos.cn>
>>> ---
>>>   tools/lib/bpf/libbpf.c | 47 ++++++++++++++++++++++++++++++++++++++++++
>>>   1 file changed, 47 insertions(+)
>>>
>>> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
>>> index ad1ec893b41b..6a201267fa08 100644
>>> --- a/tools/lib/bpf/libbpf.c
>>> +++ b/tools/lib/bpf/libbpf.c
>>> @@ -10421,6 +10421,50 @@ struct kprobe_multi_resolve {
>>>       size_t cnt;
>>>   };
>>> +static bool filter_available_function(const char *name)
>>> +{
>>> +    char addr_range[256];
>>> +    char sym_name[256];
>>> +    FILE *f;
>>> +    int ret;
>>> +
>>> +    f = fopen("/sys/kernel/debug/kprobes/blacklist", "r");
>>> +    if (!f)
>>> +        goto avail_filter;
>>> +
>>> +    while (true) {
>>> +        ret = fscanf(f, "%s %s%*[^\n]\n", addr_range, sym_name);
>>> +        if (ret == EOF && feof(f))
>>> +            break;
>>> +        if (ret != 2)
>>> +            break;
>>> +        if (!strcmp(name, sym_name)) {
>>> +            fclose(f);
>>> +            return false;
>>> +        }
>>> +    }
>>> +    fclose(f);
>>
>> so available_filter_functions already contains all traceable symbols
>> for kprobe_multi/fprobe
>>
>> kprobes/blacklist is kprobe specific and does not apply to fprobe,
>> is there a crash when attaching function from kprobes/blacklist ?
> 
> No, I haven't got crash before, Simply because BCC's kprobe_exists has
> implemented it so I added this, Yes, I also don't think 
> kprobes/blacklist will affect FPROBE, so I will remove it.
> 
>>
>>> +
>>> +avail_filter:
>>> +    f = 
>>> fopen("/sys/kernel/debug/tracing/available_filter_functions", "r");
>>> +    if (!f)
>>> +        return true;
>>> +
>>> +    while (true) {
>>> +        ret = fscanf(f, "%s%*[^\n]\n", sym_name);
>>> +        if (ret == EOF && feof(f))
>>> +            break;
>>> +        if (ret != 1)
>>> +            break;
>>> +        if (!strcmp(name, sym_name)) {
>>> +            fclose(f);
>>> +            return true;
>>> +        }
>>> +    }
>>> +    fclose(f);
>>> +    return false;
>>> +}
>>> +
>>>   static int
>>>   resolve_kprobe_multi_cb(unsigned long long sym_addr, char sym_type,
>>>               const char *sym_name, void *ctx)
>>> @@ -10431,6 +10475,9 @@ resolve_kprobe_multi_cb(unsigned long long 
>>> sym_addr, char sym_type,
>>>       if (!glob_match(sym_name, res->pattern))
>>>           return 0;
>>> +    if (!filter_available_function(sym_name))
>>> +        return 0;
>>
>> I think it'd be better to parse available_filter_functions directly
>> for kprobe_multi instead of filtering out kallsyms entries
>>
>> we could add libbpf_available_filter_functions_parse function with
>> similar callback to go over available_filter_functions file
>>
> 
> Sure, if available_filter_functions not found, fallback to /proc/kallsyms.
> 

Um.

It is difficult to judge available_filter_functions directly, because we
not only need the function name, but also obtain its address and other
information, but we can indeed obtain the function set from
available_filter_functions first, and then obtain the function address
from /proc/kallsyms. which will be slightly faster than reading
available_filter_functions later, because if this function does not
exist in available_filter_functions, it will take a long time to read
the entire file.

Of course, it would be better if the kernel directly provided an
available_filter_functions -like file containing function address
information.

-- 
Jackie Liu