Re: [RFC PATCH v2 0/8] fprobe: Introduce fprobe function entry/exit probe

Jiri Olsa
2022-01-18T14:39:04Z

On Fri, Jan 14, 2022 at 05:08:32PM -0800, Andrii Nakryiko wrote:
> On Wed, Jan 12, 2022 at 6:02 AM Masami Hiramatsu <mhiramat@kernel.org> wrote:
> >
> > Hi Jiri and Alexei,
> >
> > Here is the 2nd version of fprobe. This version uses the
> > ftrace_set_filter_ips() for reducing the registering overhead.
> > Note that this also drops per-probe point private data, which
> > is not used anyway.
>
> This per-probe private data is necessary for the feature called BPF
> cookie, in which each attachment has a unique user-provided u64 value
> associated to it, accessible at runtime through
> bpf_get_attach_cookie() helper. One way or another we'll need to
> support this to make these multi-attach BPF programs really useful for
> generic tracing applications.
>
> Jiri,
>
> We've discussed with Alexei this week how cookies can be supported for
> multi-attach fentry (where it seems even more challenging than in
> kprobe case), and agreed on rather simple solution, which roughly goes
> like this. When multi-attaching either fentry/fexit program, save
> sorted array of IP addresses and then sorted in the same order as IPs
> list of u64 cookies. At runtime, bpf_get_attach_cookie() helper should
> somehow get access to these two arrays and functions IP (that we
> already have with bpf_get_func_ip()), perform binary search and locate
> necessary cookie. This offloads the overhead of finding this cookie to
> actual call site of bpf_get_attach_cookie() (and it's a log(N), so not
> bad at all, especially if BPF program can be optimized to call this
> helper just once).
>
> I think something like that should be doable for Masami's fprobe-based
> multi-attach kprobes, right? That would allow to have super-fast
> attachment, but still support BPF cookie per each individual IP/kernel
> function attachment. I haven't looked at code thoroughly, though,
> please let me know if I'm missing something fundamental.

ok, that seems doable, we should be able to get the link struct
in bpf_get_attach_cookie_trace and reach both ips and cookies

jirka

>
> >
> > This introduces the fprobe, the function entry/exit probe with
> > multiple probe point support. This also introduces the rethook
> > for hooking function return as same as kretprobe does. This
> > abstraction will help us to generalize the fgraph tracer,
> > because we can just switch it from rethook in fprobe, depending
> > on the kernel configuration.
> >
> > The patch [1/8] and [7/8] are from your series[1]. Other libbpf
> > patches will not be affected by this change.
> >
> > [1] https://lore.kernel.org/all/20220104080943.113249-1-jolsa@kernel.org/T/#u
> >
> > I also added an out-of-tree (just for testing) patch at the
> > end of this series ([8/8]) for adding a wildcard support to
> > the sample program. With that patch, it shows how long the
> > registration will take;
> >
> > # time insmod fprobe_example.ko symbol='btrfs_*'
> > [ 36.130947] fprobe_init: 1028 symbols found
> > [ 36.177901] fprobe_init: Planted fprobe at btrfs_*
> > real 0m 0.08s
> > user 0m 0.00s
> > sys 0m 0.07s
> >
> > Thank you,
> >
> > ---
> >
> > Jiri Olsa (2):
> > ftrace: Add ftrace_set_filter_ips function
> > bpf: Add kprobe link for attaching raw kprobes
> >
> > Masami Hiramatsu (6):
> > fprobe: Add ftrace based probe APIs
> > rethook: Add a generic return hook
> > rethook: x86: Add rethook x86 implementation
> > fprobe: Add exit_handler support
> > fprobe: Add sample program for fprobe
> > [DO NOT MERGE] Out-of-tree: Support wildcard symbol option to sample
> >
> >
> > arch/x86/Kconfig | 1
> > arch/x86/kernel/Makefile | 1
> > arch/x86/kernel/rethook.c | 115 ++++++++++++++++++++
> > include/linux/bpf_types.h | 1
> > include/linux/fprobe.h | 57 ++++++++++
> > include/linux/ftrace.h | 3 +
> > include/linux/rethook.h | 74 +++++++++++++
> > include/linux/sched.h | 3 +
> > include/uapi/linux/bpf.h | 12 ++
> > kernel/bpf/syscall.c | 195 +++++++++++++++++++++++++++++++++-
> > kernel/exit.c | 2
> > kernel/fork.c | 3 +
> > kernel/kallsyms.c | 1
> > kernel/trace/Kconfig | 22 ++++
> > kernel/trace/Makefile | 2
> > kernel/trace/fprobe.c | 168 +++++++++++++++++++++++++++++
> > kernel/trace/ftrace.c | 54 ++++++++-
> > kernel/trace/rethook.c | 226 +++++++++++++++++++++++++++++++++++++++
> > samples/Kconfig | 7 +
> > samples/Makefile | 1
> > samples/fprobe/Makefile | 3 +
> > samples/fprobe/fprobe_example.c | 154 +++++++++++++++++++++++++++
> > tools/include/uapi/linux/bpf.h | 12 ++
> > 23 files changed, 1103 insertions(+), 14 deletions(-)
> > create mode 100644 arch/x86/kernel/rethook.c
> > create mode 100644 include/linux/fprobe.h
> > create mode 100644 include/linux/rethook.h
> > create mode 100644 kernel/trace/fprobe.c
> > create mode 100644 kernel/trace/rethook.c
> > create mode 100644 samples/fprobe/Makefile
> > create mode 100644 samples/fprobe/fprobe_example.c
> >
> > --
> > Masami Hiramatsu (Linaro) <mhiramat@kernel.org>
>