From: Jiri Olsa <jolsa@kernel.org>
To: Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>
Cc: bpf@vger.kernel.org, linux-trace-kernel@vger.kernel.org,
Martin KaFai Lau <kafai@fb.com>,
Eduard Zingerman <eddyz87@gmail.com>,
Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
Menglong Dong <menglong8.dong@gmail.com>,
Steven Rostedt <rostedt@kernel.org>
Subject: [RFC bpf-next 00/12] bpf: tracing_multi link
Date: Tue, 3 Feb 2026 10:38:07 +0100 [thread overview]
Message-ID: <20260203093819.2105105-1-jolsa@kernel.org> (raw)
hi,
as an option to Meglong's change [1] I'm sending proposal for tracing_multi
link that does not add static trampoline but attaches program to all needed
trampolines.
This approach keeps the same performance but has some drawbacks:
- when attaching 20k functions we allocate and attach 20k trampolines
- during attachment we hold each trampoline mutex, so for above
20k functions we will hold 20k mutexes during the attachment,
should be very prone to deadlock, but haven't hit it yet
I was hoping we'd find some common solution, but it looks like it's either
static trampoline with performance penalty or having troubles described
above but keeping the current trampoline performance.
It looks the trampoline allocations/generation might not be big a problem
and I'll try to find a solution for holding that many mutexes. If there's
no better solution I think having one read/write mutex for tracing multi
link attach/detach should work.
We'd like to use trampolines instead of kprobes for the performance gains,
so naturally we want to keep the same performance even when it's attached
through tracing multi link.
thoughts? thanks,
jirka
[1] https://lore.kernel.org/bpf/20250703121521.1874196-1-dongml2@chinatelecom.cn/
---
Jiri Olsa (12):
ftrace: Add ftrace_hash_count function
bpf: Add struct bpf_trampoline_ops object
bpf: Add struct bpf_struct_ops_tramp_link object
bpf: Add struct bpf_tramp_node object
bpf: Add multi tracing attach types
bpf: Add bpf_trampoline_multi_attach/detach functions
bpf: Add support to create tracing multi link
libbpf: Add btf__find_by_glob_kind function
libbpf: Add support to create tracing multi link
selftests/bpf: Add fentry tracing multi func test
selftests/bpf: Add fentry intersected tracing multi func test
selftests/bpf: Add tracing multi benchmark test
arch/arm64/net/bpf_jit_comp.c | 58 +++++++--------
arch/s390/net/bpf_jit_comp.c | 42 +++++------
arch/x86/net/bpf_jit_comp.c | 54 +++++++-------
include/linux/bpf.h | 74 +++++++++++++------
include/linux/ftrace.h | 1 +
include/linux/trace_events.h | 6 ++
include/uapi/linux/bpf.h | 7 ++
kernel/bpf/bpf_struct_ops.c | 39 +++++-----
kernel/bpf/syscall.c | 62 +++++++++++-----
kernel/bpf/trampoline.c | 340 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----------------
kernel/bpf/verifier.c | 8 ++-
kernel/trace/bpf_trace.c | 105 +++++++++++++++++++++++++++
kernel/trace/ftrace.c | 14 ++--
net/bpf/bpf_dummy_struct_ops.c | 23 +++---
net/bpf/test_run.c | 2 +
tools/include/uapi/linux/bpf.h | 7 ++
tools/lib/bpf/bpf.c | 7 ++
tools/lib/bpf/bpf.h | 4 ++
tools/lib/bpf/btf.c | 41 +++++++++++
tools/lib/bpf/btf.h | 3 +
tools/lib/bpf/libbpf.c | 87 +++++++++++++++++++++++
tools/lib/bpf/libbpf.h | 14 ++++
tools/lib/bpf/libbpf.map | 1 +
tools/testing/selftests/bpf/Makefile | 3 +-
tools/testing/selftests/bpf/prog_tests/tracing_multi.c | 363 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
tools/testing/selftests/bpf/progs/tracing_multi_check.c | 132 ++++++++++++++++++++++++++++++++++
tools/testing/selftests/bpf/progs/tracing_multi_fentry.c | 39 ++++++++++
27 files changed, 1319 insertions(+), 217 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/tracing_multi.c
create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_check.c
create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_fentry.c
next reply other threads:[~2026-02-03 9:38 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-03 9:38 Jiri Olsa [this message]
2026-02-03 9:38 ` [RFC bpf-next 01/12] ftrace: Add ftrace_hash_count function Jiri Olsa
2026-02-03 15:40 ` Steven Rostedt
2026-02-04 12:06 ` Jiri Olsa
2026-02-03 9:38 ` [RFC bpf-next 02/12] bpf: Add struct bpf_trampoline_ops object Jiri Olsa
2026-02-03 9:38 ` [RFC bpf-next 03/12] bpf: Add struct bpf_struct_ops_tramp_link object Jiri Olsa
2026-02-03 9:38 ` [RFC bpf-next 04/12] bpf: Add struct bpf_tramp_node object Jiri Olsa
2026-02-04 19:00 ` Andrii Nakryiko
2026-02-05 8:57 ` Jiri Olsa
2026-02-05 22:27 ` Andrii Nakryiko
2026-02-06 8:27 ` Jiri Olsa
2026-02-03 9:38 ` [RFC bpf-next 05/12] bpf: Add multi tracing attach types Jiri Olsa
2026-02-03 10:13 ` bot+bpf-ci
2026-02-17 22:05 ` Jiri Olsa
2026-02-04 2:20 ` Leon Hwang
2026-02-04 12:41 ` Jiri Olsa
2026-02-03 9:38 ` [RFC bpf-next 06/12] bpf: Add bpf_trampoline_multi_attach/detach functions Jiri Olsa
2026-02-03 10:14 ` bot+bpf-ci
2026-02-17 22:05 ` Jiri Olsa
2026-02-05 9:16 ` Menglong Dong
2026-02-05 13:45 ` Jiri Olsa
2026-02-11 8:04 ` Menglong Dong
2026-02-03 9:38 ` [RFC bpf-next 07/12] bpf: Add support to create tracing multi link Jiri Olsa
2026-02-03 10:13 ` bot+bpf-ci
2026-02-17 22:05 ` Jiri Olsa
2026-02-04 19:05 ` Andrii Nakryiko
2026-02-05 8:55 ` Jiri Olsa
2026-02-03 9:38 ` [RFC bpf-next 08/12] libbpf: Add btf__find_by_glob_kind function Jiri Olsa
2026-02-03 10:14 ` bot+bpf-ci
2026-02-04 19:04 ` Andrii Nakryiko
2026-02-05 8:57 ` Jiri Olsa
2026-02-05 22:45 ` Andrii Nakryiko
2026-02-06 8:43 ` Jiri Olsa
2026-02-06 16:58 ` Andrii Nakryiko
2026-02-03 9:38 ` [RFC bpf-next 09/12] libbpf: Add support to create tracing multi link Jiri Olsa
2026-02-03 10:14 ` bot+bpf-ci
2026-02-17 22:05 ` Jiri Olsa
2026-02-04 19:05 ` Andrii Nakryiko
2026-02-17 22:06 ` Jiri Olsa
2026-02-03 9:38 ` [RFC bpf-next 10/12] selftests/bpf: Add fentry tracing multi func test Jiri Olsa
2026-02-03 10:13 ` bot+bpf-ci
2026-02-17 22:06 ` Jiri Olsa
2026-02-03 9:38 ` [RFC bpf-next 11/12] selftests/bpf: Add fentry intersected " Jiri Olsa
2026-02-03 9:38 ` [RFC bpf-next 12/12] selftests/bpf: Add tracing multi benchmark test Jiri Olsa
2026-02-03 10:13 ` bot+bpf-ci
2026-02-17 22:06 ` Jiri Olsa
2026-02-03 23:17 ` [RFC bpf-next 00/12] bpf: tracing_multi link Alexei Starovoitov
2026-02-04 12:36 ` Jiri Olsa
2026-02-04 16:06 ` Alexei Starovoitov
2026-02-05 8:55 ` Jiri Olsa
2026-02-05 15:55 ` Alexei Starovoitov
2026-02-06 8:18 ` Jiri Olsa
2026-02-06 17:03 ` Andrii Nakryiko
2026-02-08 20:54 ` Jiri Olsa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260203093819.2105105-1-jolsa@kernel.org \
--to=jolsa@kernel.org \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=eddyz87@gmail.com \
--cc=kafai@fb.com \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=menglong8.dong@gmail.com \
--cc=rostedt@kernel.org \
--cc=songliubraving@fb.com \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox