* [PATCH bpf-next 1/2] Revert "bpf: Add support to inline bpf_get_func_ip helper on x86"
2022-03-21 7:01 [PATCH bpf-next 0/2] bpf: Fix kprobe_multi return probe backtrace Jiri Olsa
@ 2022-03-21 7:01 ` Jiri Olsa
2022-03-21 7:01 ` [PATCH bpf-next 2/2] bpf: Fix kprobe_multi return probe backtrace Jiri Olsa
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Jiri Olsa @ 2022-03-21 7:01 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Masami Hiramatsu
Cc: netdev, bpf, lkml, Martin KaFai Lau, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Steven Rostedt
This reverts commit 97ee4d20ee67eb462581a7af01442de6586e390b.
Following change is adding more complexity to bpf_get_func_ip
helper for kprobe_multi programs, which can't be inlined easily.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
kernel/bpf/verifier.c | 21 +--------------------
kernel/trace/bpf_trace.c | 1 -
2 files changed, 1 insertion(+), 21 deletions(-)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 0287176bfe9a..cf92f9c01556 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -13678,7 +13678,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
continue;
}
- /* Implement tracing bpf_get_func_ip inline. */
+ /* Implement bpf_get_func_ip inline. */
if (prog_type == BPF_PROG_TYPE_TRACING &&
insn->imm == BPF_FUNC_get_func_ip) {
/* Load IP address from ctx - 16 */
@@ -13693,25 +13693,6 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
continue;
}
-#ifdef CONFIG_X86
- /* Implement kprobe_multi bpf_get_func_ip inline. */
- if (prog_type == BPF_PROG_TYPE_KPROBE &&
- eatype == BPF_TRACE_KPROBE_MULTI &&
- insn->imm == BPF_FUNC_get_func_ip) {
- /* Load IP address from ctx (struct pt_regs) ip */
- insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1,
- offsetof(struct pt_regs, ip));
-
- new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, 1);
- if (!new_prog)
- return -ENOMEM;
-
- env->prog = prog = new_prog;
- insn = new_prog->insnsi + i + delta;
- continue;
- }
-#endif
-
patch_call_imm:
fn = env->ops->get_func_proto(insn->imm, env->prog);
/* all functions that have prototype and verifier allowed
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 9a7b6be655e4..52c2998e1dc3 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -1042,7 +1042,6 @@ static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = {
BPF_CALL_1(bpf_get_func_ip_kprobe_multi, struct pt_regs *, regs)
{
- /* This helper call is inlined by verifier on x86. */
return instruction_pointer(regs);
}
--
2.35.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH bpf-next 2/2] bpf: Fix kprobe_multi return probe backtrace
2022-03-21 7:01 [PATCH bpf-next 0/2] bpf: Fix kprobe_multi return probe backtrace Jiri Olsa
2022-03-21 7:01 ` [PATCH bpf-next 1/2] Revert "bpf: Add support to inline bpf_get_func_ip helper on x86" Jiri Olsa
@ 2022-03-21 7:01 ` Jiri Olsa
2022-03-21 14:00 ` [PATCH bpf-next 0/2] " patchwork-bot+netdevbpf
2022-03-22 0:53 ` Masami Hiramatsu
3 siblings, 0 replies; 5+ messages in thread
From: Jiri Olsa @ 2022-03-21 7:01 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Masami Hiramatsu
Cc: netdev, bpf, lkml, Martin KaFai Lau, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Steven Rostedt
Andrii reported that backtraces from kprobe_multi program attached
as return probes are not complete and showing just initial entry [1].
It's caused by changing registers to have original function ip address
as instruction pointer even for return probe, which will screw backtrace
from return probe.
This change keeps registers intact and store original entry ip and
link address on the stack in bpf_kprobe_multi_run_ctx struct, where
bpf_get_func_ip and bpf_get_attach_cookie helpers for kprobe_multi
programs can find it.
[1] https://lore.kernel.org/bpf/CAEf4BzZDDqK24rSKwXNp7XL3ErGD4bZa1M6c_c4EvDSt3jrZcg@mail.gmail.com/T/#m8d1301c0ea0892ddf9dc6fba57a57b8cf11b8c51
Fixes: ca74823c6e16 ("bpf: Add cookie support to programs attached with kprobe multi link")
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
kernel/trace/bpf_trace.c | 67 ++++++++++++++++++++++------------------
1 file changed, 37 insertions(+), 30 deletions(-)
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 52c2998e1dc3..172ef545730d 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -80,7 +80,8 @@ u64 bpf_get_stack(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
static int bpf_btf_printf_prepare(struct btf_ptr *ptr, u32 btf_ptr_size,
u64 flags, const struct btf **btf,
s32 *btf_id);
-static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx, u64 ip);
+static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
+static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
/**
* trace_call_bpf - invoke BPF program
@@ -1042,7 +1043,7 @@ static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = {
BPF_CALL_1(bpf_get_func_ip_kprobe_multi, struct pt_regs *, regs)
{
- return instruction_pointer(regs);
+ return bpf_kprobe_multi_entry_ip(current->bpf_ctx);
}
static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe_multi = {
@@ -1054,7 +1055,7 @@ static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe_multi = {
BPF_CALL_1(bpf_get_attach_cookie_kprobe_multi, struct pt_regs *, regs)
{
- return bpf_kprobe_multi_cookie(current->bpf_ctx, instruction_pointer(regs));
+ return bpf_kprobe_multi_cookie(current->bpf_ctx);
}
static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = {
@@ -2219,15 +2220,16 @@ struct bpf_kprobe_multi_link {
struct bpf_link link;
struct fprobe fp;
unsigned long *addrs;
- /*
- * The run_ctx here is used to get struct bpf_kprobe_multi_link in
- * get_attach_cookie helper, so it can't be used to store data.
- */
- struct bpf_run_ctx run_ctx;
u64 *cookies;
u32 cnt;
};
+struct bpf_kprobe_multi_run_ctx {
+ struct bpf_run_ctx run_ctx;
+ struct bpf_kprobe_multi_link *link;
+ unsigned long entry_ip;
+};
+
static void bpf_kprobe_multi_link_release(struct bpf_link *link)
{
struct bpf_kprobe_multi_link *kmulti_link;
@@ -2281,18 +2283,21 @@ static int bpf_kprobe_multi_cookie_cmp(const void *a, const void *b, const void
return __bpf_kprobe_multi_cookie_cmp(a, b);
}
-static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx, u64 ip)
+static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx)
{
+ struct bpf_kprobe_multi_run_ctx *run_ctx;
struct bpf_kprobe_multi_link *link;
+ u64 *cookie, entry_ip;
unsigned long *addr;
- u64 *cookie;
if (WARN_ON_ONCE(!ctx))
return 0;
- link = container_of(ctx, struct bpf_kprobe_multi_link, run_ctx);
+ run_ctx = container_of(current->bpf_ctx, struct bpf_kprobe_multi_run_ctx, run_ctx);
+ link = run_ctx->link;
if (!link->cookies)
return 0;
- addr = bsearch(&ip, link->addrs, link->cnt, sizeof(ip),
+ entry_ip = run_ctx->entry_ip;
+ addr = bsearch(&entry_ip, link->addrs, link->cnt, sizeof(entry_ip),
__bpf_kprobe_multi_cookie_cmp);
if (!addr)
return 0;
@@ -2300,10 +2305,22 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx, u64 ip)
return *cookie;
}
+static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx)
+{
+ struct bpf_kprobe_multi_run_ctx *run_ctx;
+
+ run_ctx = container_of(current->bpf_ctx, struct bpf_kprobe_multi_run_ctx, run_ctx);
+ return run_ctx->entry_ip;
+}
+
static int
kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
- struct pt_regs *regs)
+ unsigned long entry_ip, struct pt_regs *regs)
{
+ struct bpf_kprobe_multi_run_ctx run_ctx = {
+ .link = link,
+ .entry_ip = entry_ip,
+ };
struct bpf_run_ctx *old_run_ctx;
int err;
@@ -2314,7 +2331,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
migrate_disable();
rcu_read_lock();
- old_run_ctx = bpf_set_run_ctx(&link->run_ctx);
+ old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx);
err = bpf_prog_run(link->link.prog, regs);
bpf_reset_run_ctx(old_run_ctx);
rcu_read_unlock();
@@ -2329,24 +2346,10 @@ static void
kprobe_multi_link_handler(struct fprobe *fp, unsigned long entry_ip,
struct pt_regs *regs)
{
- unsigned long saved_ip = instruction_pointer(regs);
struct bpf_kprobe_multi_link *link;
- /*
- * Because fprobe's regs->ip is set to the next instruction of
- * dynamic-ftrace instruction, correct entry ip must be set, so
- * that the bpf program can access entry address via regs as same
- * as kprobes.
- *
- * Both kprobe and kretprobe see the entry ip of traced function
- * as instruction pointer.
- */
- instruction_pointer_set(regs, entry_ip);
-
link = container_of(fp, struct bpf_kprobe_multi_link, fp);
- kprobe_multi_link_prog_run(link, regs);
-
- instruction_pointer_set(regs, saved_ip);
+ kprobe_multi_link_prog_run(link, entry_ip, regs);
}
static int
@@ -2513,7 +2516,11 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
{
return -EOPNOTSUPP;
}
-static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx, u64 ip)
+static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx)
+{
+ return 0;
+}
+static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx)
{
return 0;
}
--
2.35.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH bpf-next 0/2] bpf: Fix kprobe_multi return probe backtrace
2022-03-21 7:01 [PATCH bpf-next 0/2] bpf: Fix kprobe_multi return probe backtrace Jiri Olsa
` (2 preceding siblings ...)
2022-03-21 14:00 ` [PATCH bpf-next 0/2] " patchwork-bot+netdevbpf
@ 2022-03-22 0:53 ` Masami Hiramatsu
3 siblings, 0 replies; 5+ messages in thread
From: Masami Hiramatsu @ 2022-03-22 0:53 UTC (permalink / raw)
To: Jiri Olsa
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, netdev, bpf,
lkml, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Steven Rostedt
On Mon, 21 Mar 2022 08:01:11 +0100
Jiri Olsa <jolsa@kernel.org> wrote:
> hi,
> Andrii reported that backtraces from kprobe_multi program attached
> as return probes are not complete and showing just initial entry [1].
>
> Sending the fix together with bpf_get_func_ip inline revert, which is
> no longer suitable.
OK, this series looks good to me.
Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org>
Thank you,
>
> thanks,
> jirka
>
>
> ---
> [1] https://lore.kernel.org/bpf/CAEf4BzZDDqK24rSKwXNp7XL3ErGD4bZa1M6c_c4EvDSt3jrZcg@mail.gmail.com/T/#m8d1301c0ea0892ddf9dc6fba57a57b8cf11b8c51
>
> Jiri Olsa (2):
> Revert "bpf: Add support to inline bpf_get_func_ip helper on x86"
> bpf: Fix kprobe_multi return probe backtrace
>
> kernel/bpf/verifier.c | 21 +--------------------
> kernel/trace/bpf_trace.c | 68 +++++++++++++++++++++++++++++++++++++-------------------------------
> 2 files changed, 38 insertions(+), 51 deletions(-)
--
Masami Hiramatsu <mhiramat@kernel.org>
^ permalink raw reply [flat|nested] 5+ messages in thread