Re: [PATCHv6 perf/core 10/22] uprobes/x86: Add support to optimize uprobes

[Jiri Olsa <olsajiri@gmail.com>]

ping, thanks

On Mon, Jul 28, 2025 at 11:34:56PM +0200, Jiri Olsa wrote:
> On Fri, Jul 25, 2025 at 07:13:18PM +0900, Masami Hiramatsu wrote:
> > On Sun, 20 Jul 2025 13:21:20 +0200
> > Jiri Olsa <jolsa@kernel.org> wrote:
> > 
> > > Putting together all the previously added pieces to support optimized
> > > uprobes on top of 5-byte nop instruction.
> > > 
> > > The current uprobe execution goes through following:
> > > 
> > >   - installs breakpoint instruction over original instruction
> > >   - exception handler hit and calls related uprobe consumers
> > >   - and either simulates original instruction or does out of line single step
> > >     execution of it
> > >   - returns to user space
> > > 
> > > The optimized uprobe path does following:
> > > 
> > >   - checks the original instruction is 5-byte nop (plus other checks)
> > >   - adds (or uses existing) user space trampoline with uprobe syscall
> > >   - overwrites original instruction (5-byte nop) with call to user space
> > >     trampoline
> > >   - the user space trampoline executes uprobe syscall that calls related uprobe
> > >     consumers
> > >   - trampoline returns back to next instruction
> > > 
> > > This approach won't speed up all uprobes as it's limited to using nop5 as
> > > original instruction, but we plan to use nop5 as USDT probe instruction
> > > (which currently uses single byte nop) and speed up the USDT probes.
> > > 
> > > The arch_uprobe_optimize triggers the uprobe optimization and is called after
> > > first uprobe hit. I originally had it called on uprobe installation but then
> > > it clashed with elf loader, because the user space trampoline was added in a
> > > place where loader might need to put elf segments, so I decided to do it after
> > > first uprobe hit when loading is done.
> > > 
> > > The uprobe is un-optimized in arch specific set_orig_insn call.
> > > 
> > > The instruction overwrite is x86 arch specific and needs to go through 3 updates:
> > > (on top of nop5 instruction)
> > > 
> > >   - write int3 into 1st byte
> > >   - write last 4 bytes of the call instruction
> > >   - update the call instruction opcode
> > > 
> > > And cleanup goes though similar reverse stages:
> > > 
> > >   - overwrite call opcode with breakpoint (int3)
> > >   - write last 4 bytes of the nop5 instruction
> > >   - write the nop5 first instruction byte
> > > 
> > > We do not unmap and release uprobe trampoline when it's no longer needed,
> > > because there's no easy way to make sure none of the threads is still
> > > inside the trampoline. But we do not waste memory, because there's just
> > > single page for all the uprobe trampoline mappings.
> > > 
> > > We do waste frame on page mapping for every 4GB by keeping the uprobe
> > > trampoline page mapped, but that seems ok.
> > > 
> > > We take the benefit from the fact that set_swbp and set_orig_insn are
> > > called under mmap_write_lock(mm), so we can use the current instruction
> > > as the state the uprobe is in - nop5/breakpoint/call trampoline -
> > > and decide the needed action (optimize/un-optimize) based on that.
> > > 
> > > Attaching the speed up from benchs/run_bench_uprobes.sh script:
> > > 
> > > current:
> > >         usermode-count :  152.604 ± 0.044M/s
> > >         syscall-count  :   13.359 ± 0.042M/s
> > > -->     uprobe-nop     :    3.229 ± 0.002M/s
> > >         uprobe-push    :    3.086 ± 0.004M/s
> > >         uprobe-ret     :    1.114 ± 0.004M/s
> > >         uprobe-nop5    :    1.121 ± 0.005M/s
> > >         uretprobe-nop  :    2.145 ± 0.002M/s
> > >         uretprobe-push :    2.070 ± 0.001M/s
> > >         uretprobe-ret  :    0.931 ± 0.001M/s
> > >         uretprobe-nop5 :    0.957 ± 0.001M/s
> > > 
> > > after the change:
> > >         usermode-count :  152.448 ± 0.244M/s
> > >         syscall-count  :   14.321 ± 0.059M/s
> > >         uprobe-nop     :    3.148 ± 0.007M/s
> > >         uprobe-push    :    2.976 ± 0.004M/s
> > >         uprobe-ret     :    1.068 ± 0.003M/s
> > > -->     uprobe-nop5    :    7.038 ± 0.007M/s
> > >         uretprobe-nop  :    2.109 ± 0.004M/s
> > >         uretprobe-push :    2.035 ± 0.001M/s
> > >         uretprobe-ret  :    0.908 ± 0.001M/s
> > >         uretprobe-nop5 :    3.377 ± 0.009M/s
> > > 
> > > I see bit more speed up on Intel (above) compared to AMD. The big nop5
> > > speed up is partly due to emulating nop5 and partly due to optimization.
> > > 
> > > The key speed up we do this for is the USDT switch from nop to nop5:
> > >         uprobe-nop     :    3.148 ± 0.007M/s
> > >         uprobe-nop5    :    7.038 ± 0.007M/s
> > > 
> > 
> > This also looks good to me.
> > 
> > Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
> 
> thanks!
> 
> Peter, do you have more comments?
> 
> thanks,
> jirka