All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH bpf-next v3 0/3] bpf: Use an on-stack pt_regs in the raw tracepoint helpers
@ 2026-11-20 10:59 ` Mike Bazov via B4 Relay
  0 siblings, 0 replies; 9+ messages in thread
From: Mike Bazov @ 2026-11-20 10:59 UTC (permalink / raw)
  To: Song Liu, Jiri Olsa, Ihor Solodrai, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Eduard Zingerman,
	Kumar Kartikeya Dwivedi, Martin KaFai Lau, Yonghong Song,
	Emil Tsalapatis, KP Singh, Matt Bobrowski, Steven Rostedt,
	Masami Hiramatsu, Mathieu Desnoyers
  Cc: bpf, linux-kernel, linux-trace-kernel, Mike Bazov

bpf_perf_event_output(), bpf_get_stackid() and bpf_get_stack() of raw
tracepoint and tracing programs share three per-CPU pt_regs slots,
picked by a nest counter. These programs can be preempted, and tasks
then leave the slots out of order, so a sample or stack trace of one
task gets the registers of another. Patch 3 fills in a pt_regs on the
stack of each helper instead and removes the slots.

The slots also limited how deep these helpers nest. Two things need a
limit of their own without them, and both can already be hit today
without raw tracepoint programs:

- bpf_perf_event_output() WARNs when entered a fourth time on a CPU.

- Build ID collection in bpf_get_stack()/bpf_get_stackid() and
  bpf_find_vma() take mmap_read_trylock(), which fires the mmap_lock
  tracepoints. Programs on them that call these helpers nest into each
  other as deep as there are programs. Patch 2 limits it to three
  levels.

Results on bpf-next 0fb18ce8a2e9, x86-64, debug config, CONFIG_PREEMPT=y,
preempt=full. "base" is that tree, "v2" adds only patch 3, "v3" all
three.

The slot race, 5 runs of 10s each. Two raw tracepoint programs on
sys_enter call the helper, two threads make syscalls on CPU 0 and a
SCHED_FIFO thread preempts them every 20us. For bpf_perf_event_output()
the reproducer reads the return address of each program's helper call
from its JIT image and checks the IP of every sample; bpf_get_stack()
and bpf_get_stackid() get an empty kernel stack instead, because the
unwinder looks for the other task's sp on the current stack:

                                     base        v3
  bpf_perf_event_output, wrong IP    7-18        0      per program
  bpf_get_stack, empty stacks        362-535     0
  bpf_get_stackid, empty stacks      168-364     0

The bpf_get_stackid() case with bpftrace, whose kstack calls it; @a[]
and @b[] count the empty stacks:

  bpftrace -e '
  rawtracepoint:sys_enter /comm == "dd"/ { @a[kstack(1)] = count(); }
  rawtracepoint:sys_enter /comm == "dd"/ { @b[kstack(1)] = count(); }' &
  taskset -c 0 dd if=/dev/zero of=/dev/null bs=1 &
  taskset -c 0 dd if=/dev/zero of=/dev/null bs=1 &
  taskset -c 0 chrt -f 1 perl -e 'select(undef, undef, undef, 2e-5) while 1' &

Nesting. For the WARN, a raw tracepoint or kprobe program calls
bpf_perf_event_output(), and three fentry programs on
perf_event_output() call it too. For mmap_read_trylock(), six programs
on mmap_lock_acquire_returned call bpf_get_stack() with build IDs, or
bpf_find_vma(), while the process does 1000 mmap()/munmap():

                                         base     v2       v3
  WARN, started from raw tracepoint      no       yes      no
  WARN, started from kprobe              yes      -        no
  bpf_get_stack, build IDs: depth        4        6        4
                            calls        1.5M     5.9M     1.5M
  bpf_find_vma: depth                    6        6        4
                calls                    5.9M     5.9M     1.5M

Depth counts the programs on the stack; at 4, the last one gets -EBUSY
or the fallback to IPs. The kprobe case wasn't run on v2, which doesn't
change that path.

---
Changes in v3:
- New patch 1: don't WARN when bpf_perf_event_output() nests too deep
  (reported by the BPF CI bot).
- New patch 2: limit nesting of mmap_read_trylock() in BPF helpers
  (per Alexei's suggestion).
- Patch 3: correct the commit message, bpf_perf_event_output() keeps
  its own nesting limit.
- Link to v2: https://lore.kernel.org/r/20261119-bpf-raw-tp-regs-preempt-v2-1-e1e1b90857d6@gmail.com

Changes in v2:
- Use a pt_regs on the stack of each helper and remove the per-CPU
  slots, instead of disabling preemption while a slot is in use
  (per Andrii's suggestion).
- Link to v1: https://lore.kernel.org/r/20261116-bpf-raw-tp-regs-preempt-v1-1-d01b6dd8573c@gmail.com

---
Mike Bazov (3):
      bpf: Don't WARN when bpf_perf_event_output() nests too deep
      bpf: Limit nesting of mmap_read_trylock() in BPF helpers
      bpf: Use an on-stack pt_regs in the raw tracepoint helpers

 kernel/bpf/mmap_unlock_work.h | 16 +++++++--
 kernel/trace/bpf_trace.c      | 80 +++++++------------------------------------
 2 files changed, 26 insertions(+), 70 deletions(-)
---
base-commit: 8835113107b2cf75df33a71e265cdeec5d5d13e0
change-id: 20261116-bpf-raw-tp-regs-preempt-1f7019989ab9

Best regards,
-- 
Mike Bazov <mike.bazov@gmail.com>


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2026-11-20 11:03 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-20 10:59 [PATCH bpf-next v3 0/3] bpf: Use an on-stack pt_regs in the raw tracepoint helpers Mike Bazov
2026-11-20 10:59 ` Mike Bazov via B4 Relay
2026-11-20 10:59 ` [PATCH bpf-next v3 1/3] bpf: Don't WARN when bpf_perf_event_output() nests too deep Mike Bazov
2026-11-20 10:59   ` Mike Bazov via B4 Relay
2026-11-20 11:03   ` sashiko-bot
2026-11-20 10:59 ` [PATCH bpf-next v3 2/3] bpf: Limit nesting of mmap_read_trylock() in BPF helpers Mike Bazov
2026-11-20 10:59   ` Mike Bazov via B4 Relay
2026-11-20 10:59 ` [PATCH bpf-next v3 3/3] bpf: Use an on-stack pt_regs in the raw tracepoint helpers Mike Bazov
2026-11-20 10:59   ` Mike Bazov via B4 Relay

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.