Re: [RFC] systemtap: begin the process of using proper kernel APIs (part1: use kprobe symbol_name/offset instead of address)

James Bottomley
2008-07-17T14:18:27Z

On Wed, 2008-07-16 at 20:49 -0500, James Bottomley wrote:
> On Wed, 2008-07-16 at 20:05 -0400, Masami Hiramatsu wrote:
> > Hmm, here is what I got with your patch;
> > $ stap --kelf -e 'probe kernel.function("do_open"){}' -p2
> > # probes
> > kernel.function("do_open@arch/x86/kernel/apm_32.c:1557") /* pc=<do_open+0x0> */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@fs/block_dev.c:928") /* pc=<do_open+0x0> */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@fs/nfsctl.c:24") /* pc=<sys_nfsservctl+0x55> */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@ipc/mqueue.c:630") /* pc=<do_open+0x0> */ /* <- kernel.function("do_open") */
> >
> > Without your patch;
> > $ stap -e 'probe kernel.function("do_open"){}' -p2
> > # probes
> > kernel.function("do_open@arch/x86/kernel/apm_32.c:1557") /* pc=0x10382 */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@fs/block_dev.c:928") /* pc=0xa0750 */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@fs/nfsctl.c:24") /* pc=0xa6411 */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@ipc/mqueue.c:630") /* pc=0xc55a6 */ /* <- kernel.function("do_open") */
> >
> > Obviously, the 3rd "do_open" is fully inlined, so it can be
> > correctly handled by kprobes, because it has different
> > symbol(sys_nfsservctl). However, other "do_open" have
> > same symbol(do_open). these will be put on same
> > address (at the first symbol on kallsyms list).
> >
> > So, we need a bridge for the gap of function addresses
> > between kallsyms and dwarf.
>
> You mean this particular problem:
>
> hobholes:/home/jejb/git/BUILD-2.6# grep do_open /proc/kallsyms
> c01af160 t do_open
> c01d5d40 t do_open
>
> It's certainly a material defect in the current API. I'll think about
> it and see if I can come up with a solution.

OK, thought about it. There seem to be two possible solutions

1. Get systemtap always to offset from non-static functions. This
will use the standard linker to ensure uniqueness (a module
qualifier will still need to be added to the struct kprobe for
lookup, since modules can duplicate unexported kernel symbols).
2. Add the filename as a discriminator for duplicate symbols in the
kallsyms program (would still need module qualifier). This is
appealing because the path name would be printed in the kernel
trace to help with oops tracking

This is where negotiations come in. To me 2. looks to be better because
it will help us with oops tracking. On the other hand, it's usually
pretty obvious from the stack trace context which files the duplicate
symbols are actually in; what do other people think?

James