[PATCH RFCv2 00/18] uprobes: Add support to optimize usdt probes on x86_64

[Jiri Olsa <jolsa@kernel.org>]

hi,
this patchset adds support to optimize usdt probes on top of 5-byte
nop instruction.

The generic approach (optimize all uprobes) is hard due to emulating
possible multiple original instructions and its related issues. The
usdt case, which stores 5-byte nop seems much easier, so starting
with that.

The basic idea is to replace breakpoint exception with syscall which
is faster on x86_64. For more details please see changelog of patch 8.

The run_bench_uprobes.sh benchmark triggers uprobe (on top of different
original instructions) in a loop and counts how many of those happened
per second (the unit below is million loops).

There's big speed up if you consider current usdt implementation
(uprobe-nop) compared to proposed usdt (uprobe-nop5):

  # ./benchs/run_bench_uprobes.sh 

          usermode-count :  818.386 ± 1.886M/s
          syscall-count  :    8.923 ± 0.003M/s
  -->     uprobe-nop     :    3.086 ± 0.005M/s
          uprobe-push    :    2.751 ± 0.001M/s
          uprobe-ret     :    1.481 ± 0.000M/s
  -->     uprobe-nop5    :    4.016 ± 0.002M/s
          uretprobe-nop  :    1.712 ± 0.008M/s
          uretprobe-push :    1.616 ± 0.001M/s
          uretprobe-ret  :    1.052 ± 0.000M/s
          uretprobe-nop5 :    2.015 ± 0.000M/s


rfc v2 changes:
- make uretprobe work properly with optimized uprobe
- make the uprobe optimized code x86_64 specific [Peter]
- rework the verify function logic, using it now as callback
- fix find_nearest_page to include [PAGE_SIZE, ... ] area [Andrii]
- try lockless vma lookup in in_uprobe_trampoline [Peter]
- do per partes instructions update using int3 like in text_poke_bp_batch [David]
- map uprobe trampoline via single global page [Thomas]
- keep track of uprobes per mm_struct

pending todo (follow ups):
- use PROCMAP_QUERY in tests
- alloc 'struct uprobes_state' for mm_struct only when needed [Andrii]
- seccomp change for new uprobe syscall (same as for uretprobe)

thanks,
jirka


---
Jiri Olsa (18):
      uprobes: Rename arch_uretprobe_trampoline function
      uprobes: Make copy_from_page global
      uprobes: Move ref_ctr_offset update out of uprobe_write_opcode
      uprobes: Add uprobe_write function
      uprobes: Add nbytes argument to uprobe_write_opcode
      uprobes: Add orig argument to uprobe_write and uprobe_write_opcode
      uprobes: Add swbp argument to arch_uretprobe_hijack_return_addr
      uprobes/x86: Add uprobe syscall to speed up uprobe
      uprobes/x86: Add mapping for optimized uprobe trampolines
      uprobes/x86: Add mm_uprobe objects to track uprobes within mm
      uprobes/x86: Add support to emulate nop5 instruction
      uprobes/x86: Add support to optimize uprobes
      selftests/bpf: Reorg the uprobe_syscall test function
      selftests/bpf: Use 5-byte nop for x86 usdt probes
      selftests/bpf: Add uprobe/usdt syscall tests
      selftests/bpf: Add hit/attach/detach race optimized uprobe test
      selftests/bpf: Add uprobe syscall sigill signal test
      selftests/bpf: Add 5-byte nop uprobe trigger bench

 arch/arm/probes/uprobes/core.c                              |   4 +-
 arch/arm64/kernel/probes/uprobes.c                          |   2 +-
 arch/csky/kernel/probes/uprobes.c                           |   2 +-
 arch/loongarch/kernel/uprobes.c                             |   2 +-
 arch/mips/kernel/uprobes.c                                  |   2 +-
 arch/powerpc/kernel/uprobes.c                               |   2 +-
 arch/riscv/kernel/probes/uprobes.c                          |   2 +-
 arch/s390/kernel/uprobes.c                                  |   2 +-
 arch/sparc/kernel/uprobes.c                                 |   2 +-
 arch/x86/entry/syscalls/syscall_64.tbl                      |   1 +
 arch/x86/include/asm/uprobes.h                              |   6 ++
 arch/x86/kernel/uprobes.c                                   | 530 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 include/linux/syscalls.h                                    |   2 +
 include/linux/uprobes.h                                     |  23 +++-
 kernel/events/uprobes.c                                     | 147 +++++++++++++++++--------
 kernel/fork.c                                               |   1 +
 kernel/sys_ni.c                                             |   1 +
 tools/testing/selftests/bpf/bench.c                         |  12 +++
 tools/testing/selftests/bpf/benchs/bench_trigger.c          |  42 ++++++++
 tools/testing/selftests/bpf/benchs/run_bench_uprobes.sh     |   2 +-
 tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c     | 342 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c |  34 +++++-
 tools/testing/selftests/bpf/sdt.h                           |   9 +-
 23 files changed, 1093 insertions(+), 79 deletions(-)