[RFC PATCH bpf-next v1 00/11] Unwinding for synchronous callbacks

[Kumar Kartikeya Dwivedi <memxor@gmail.com>]

A callback that a helper calls cannot unwind: the helper is a C frame
in between, and it holds things only its epilogue drops.

Code that unwinds due to panics or exceptions doesn't often control
where they can be invoked, and reusing the same code in callbacks would
lead to verifier rejections since unwinding across such synchronous
callbacks is not supported.

With this series, we allow a helper gets a pad of its own. A new macro
bpf_call_cb() calls the callback from an asm goto with an extable entry
at the return address that points to a label in the helper.  unwind_exit
of the callback lands there, the helper drops what it holds, and
bpf_unwind_leave() sends it on to the pad after its own call, with the
r0 the callback unwound with. The normal path pays nothing.

To the verifier such a call is a call of something that can unwind:
unwind_goto has to follow it, and the pad gets the r0 of the callback
with its state. bpf_loop(), bpf_for_each_map_elem(), bpf_find_vma(),
bpf_user_ringbuf_drain() and the comparator of bpf_rbtree_add() are
opened up one by one. A callback may unwind with what it may store into
callback_ctx already, and the fix for the pointers that don't outlive
the helper [1] will apply to r0 as well.

Tested on x86-64 with clang, with and without kCFI, and gcc for the
objects. The arm64 objects compile, but the arm64 run is left to CI.

  [1]: https://lore.kernel.org/bpf/d2369629-846e-4fd8-b567-a0fd8863256c@linux.dev/

Kumar Kartikeya Dwivedi (11):
  bpf: Give a helper a way to call its callback with a pad
  bpf, arm64: Let the pad after a helper call have r0
  bpf: Let a callback unwind into the pad of its helper
  bpf: Give the inlined bpf_loop a pad
  bpf: Let the callback of bpf_loop unwind
  bpf: Let the callback of bpf_for_each_map_elem unwind
  bpf: Let the callback of bpf_find_vma unwind
  bpf: Let the callback of bpf_user_ringbuf_drain unwind
  bpf: Let the comparator of bpf_rbtree_add unwind
  selftests/bpf: Test callbacks that unwind
  selftests/bpf: Test what a callback unwinds with

 arch/arm64/net/bpf_jit_comp.c                 |  129 +-
 arch/x86/net/bpf_jit_comp.c                   |   25 +-
 include/linux/bpf_unwind.h                    |  282 +++++
 include/linux/bpf_verifier.h                  |    6 +-
 include/linux/filter.h                        |    1 +
 kernel/bpf/arraymap.c                         |   11 +-
 kernel/bpf/backtrack.c                        |   21 +-
 kernel/bpf/bpf_iter.c                         |   32 +-
 kernel/bpf/core.c                             |   10 +
 kernel/bpf/fixups.c                           |   43 +-
 kernel/bpf/hashtab.c                          |   26 +-
 kernel/bpf/helpers.c                          |   50 +-
 kernel/bpf/liveness.c                         |    7 +
 kernel/bpf/ringbuf.c                          |   23 +-
 kernel/bpf/task_iter.c                        |   21 +-
 kernel/bpf/verifier.c                         |  162 ++-
 .../selftests/bpf/prog_tests/unwind_drain.c   |   71 ++
 .../selftests/bpf/prog_tests/verifier.c       |    2 +
 .../selftests/bpf/progs/unwind_drain.c        |   69 ++
 .../selftests/bpf/progs/verifier_unwind.c     |    3 +-
 .../selftests/bpf/progs/verifier_unwind_cb.c  | 1064 +++++++++++++++++
 21 files changed, 1974 insertions(+), 84 deletions(-)
 create mode 100644 include/linux/bpf_unwind.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/unwind_drain.c
 create mode 100644 tools/testing/selftests/bpf/progs/unwind_drain.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_unwind_cb.c


base-commit: 4169c9c90532df9585c0a2ae513337a58f5e59ab
-- 
2.53.0