Re: [PATCH] perf probe: Fix module probing with shortname

Wangnan (F)
2015-09-23T01:15:11Z

On 2015/9/22 21:35, Arnaldo Carvalho de Melo wrote:
> Em Tue, Sep 22, 2015 at 03:34:32AM +0000, Wang Nan escreveu:
>> After commit 3d39ac538629e4f00a6e1c38d46346f1b8e69505 ("perf machine:
>> No need to have two DSOs lists"), perf probe with module short name doesn't
>> work again. For example:
>>
>> # lsmod | grep e1000e
>> e1000e 233472 0
>>
>> # cat /proc/modules | grep e1000e
>> e1000e 233472 0 - Live 0xffffffffa0073000
>>
>> # cat /proc/kallsyms | grep '\<e1000e_up\>'
>> ffffffffa0093860 t e1000e_up[e1000e]
>>
>> # perf probe -v -m e1000e --add e1000e_up
>> probe-definition(0): e1000e_up
>> symbol:e1000e_up file:(null) line:0 offset:0 return:0 lazy:(null)
>> 0 arguments
>> Failed to find module e1000e.
>> Could not open debuginfo. Try to use symbols.
>> Looking at the vmlinux_path (7 entries long)
>> Using /lib/modules/4.2.0-rc7+/build/vmlinux for symbols
>> e1000e_up is out of .text, skip it.
>> Error: Failed to add events. Reason: No such file or directory (Code: -2)
>>
>> This is caused by a misunderstood of dso->kernel in kernel_get_module_dso()
>> that, for kernel module, dso->kernel is DSO_TYPE_USER. dso->kernel is DSO_TYPE_KERNEL
>> iff dso is vmlinux.
> Kernel modules having DSO_TYPE_USER seems to be the bug, no? I'll try to
> check that...

I also noticed this problem when I working on commit
1f121b03d058dd07199d8924373d3c52a207f63b ("perf tools: Deal with kernel
module names in '[]' correctly") ;)

It should be bug, but I think fixing it is costy. Here's an assumption
that, if dso->kernel
is not zero, the dso should be vmlinux (not kernel module):

$ grep 'dso.>kernel)' ./tools/perf/ -r
./tools/perf/builtin-inject.c: if (dso->kernel)
./tools/perf/util/symbol.c: if (dso->kernel) {
./tools/perf/util/symbol-elf.c: if (dso->kernel)
./tools/perf/util/symbol-elf.c: if (remap_kernel &&
dso->kernel) {
./tools/perf/util/event.c: if (pos->dso->kernel)
./tools/perf/util/probe-event.c: if (dso->kernel)
./tools/perf/util/map.c: * map->dso->kernel) before calling
__map__is_{kernel,kmodule}())
./tools/perf/util/map.c: if (!map->dso || !map->dso->kernel) {
./tools/perf/builtin-top.c: if (!map->dso->kernel)

So care must be taken.

Another solution seems simpler: we can redefine the meaning of enum
dso_kernel_type like this:

# find ./tools/perf/ -type f | xargs -n1 sed -i
's/DSO_TYPE_USER/DSO_TYPE_NOT_VMLINUX/g'
# find ./tools/perf/ -type f | xargs -n1 sed -i
's/DSO_TYPE_KERNEL/DSO_TYPE_VMLINUX/g'
# find ./tools/perf/ -type f | xargs -n1 sed -i
's/DSO_TYPE_GUEST_KERNEL/DSO_TYPE_GUEST_VMLINUX/g'

By fixing the name of DSO_TYPE_USER, kernel module with
DSO_TYPE_NOT_VMLINUX seems
not so buggy. (Please choose a better name...)

What's your opinion?

Thank you.

> - Arnaldo
>
>> This patch fix 'perf probe -m' with an ad-hoc way.
>>
>> After this patch:
>>
>> # perf probe -v -m e1000e --add e1000e_up
>> probe-definition(0): e1000e_up
>> symbol:e1000e_up file:(null) line:0 offset:0 return:0 lazy:(null)
>> 0 arguments
>> Open Debuginfo file: /lib/modules/4.2.0-rc7+/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko
>> Try to find probe point from debuginfo.
>> Matched function: e1000e_up
>> Probe point found: e1000e_up+0
>> Found 1 probe_trace_events.
>> Opening /sys/kernel/debug/tracing//kprobe_events write=1
>> Writing event: p:probe/e1000e_up e1000e:e1000e_up+0
>> Added new event:
>> probe:e1000e_up (on e1000e_up in e1000e)
>>
>> You can now use it in all perf tools, such as:
>>
>> perf record -e probe:e1000e_up -aR sleep 1
>>
>> # perf probe -l
>> Failed to find debug information for address ffffffffa0093860
>> probe:e1000e_up (on e1000e_up in e1000e)
>>
>> Signed-off-by: Wang Nan <wangnan0@huawei.com>
>> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
>> Cc: Namhyung Kim <namhyung@kernel.org>
>> Cc: Jiri Olsa <jolsa@redhat.com>
>> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>> ---
>>
>> I think there may be other places where dso->kernel is misused.
>> machine__process_kernel_mmap_event() may be one of them. If I understand
>> correctly, 'dso->kernel && is_kernel_module(dso->long_name)' should always
>> false theoretically. However, I don't have enough time to check whether that
>> code really cause problem.
>>
>> ---
>> tools/perf/util/probe-event.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/tools/perf/util/probe-event.c b/tools/perf/util/probe-event.c
>> index 2b78e8f..c7d6d3d 100644
>> --- a/tools/perf/util/probe-event.c
>> +++ b/tools/perf/util/probe-event.c
>> @@ -270,7 +270,7 @@ static int kernel_get_module_dso(const char *module, struct dso **pdso)
>>
>> if (module) {
>> list_for_each_entry(dso, &host_machine->dsos.head, node) {
>> - if (!dso->kernel)
>> + if (dso->kernel)
>> continue;
>> if (strncmp(dso->short_name + 1, module,
>> dso->short_name_len - 2) == 0)
>> --
>> 1.8.3.4