Re: [PATCHv4 bpf-next 2/7] uprobe: Add uretprobe syscall to speed up return probe

["Edgecombe, Rick P" <rick.p.edgecombe@intel.com>]

On Fri, 2024-05-03 at 15:04 +0200, Jiri Olsa wrote:
> On Fri, May 03, 2024 at 01:34:53PM +0200, Peter Zijlstra wrote:
> > On Thu, May 02, 2024 at 02:23:08PM +0200, Jiri Olsa wrote:
> > > Adding uretprobe syscall instead of trap to speed up return probe.
> > > 
> > > At the moment the uretprobe setup/path is:
> > > 
> > >    - install entry uprobe
> > > 
> > >    - when the uprobe is hit, it overwrites probed function's return
> > > address
> > >      on stack with address of the trampoline that contains breakpoint
> > >      instruction
> > > 
> > >    - the breakpoint trap code handles the uretprobe consumers execution
> > > and
> > >      jumps back to original return address

Hi,

I worked on the x86 shadow stack support.

I didn't know uprobes did anything like this. In hindsight I should have looked
more closely. The current upstream behavior is to overwrite the return address
on the stack?

Stupid uprobes question - what is actually overwriting the return address on the
stack? Is it the kernel? If so perhaps the kernel could just update the shadow
stack at the same time.

> > > 
> > > This patch replaces the above trampoline's breakpoint instruction with new
> > > ureprobe syscall call. This syscall does exactly the same job as the trap
> > > with some more extra work:
> > > 
> > >    - syscall trampoline must save original value for rax/r11/rcx registers
> > >      on stack - rax is set to syscall number and r11/rcx are changed and
> > >      used by syscall instruction
> > > 
> > >    - the syscall code reads the original values of those registers and
> > >      restore those values in task's pt_regs area
> > > 
> > >    - only caller from trampoline exposed in '[uprobes]' is allowed,
> > >      the process will receive SIGILL signal otherwise
> > > 
> > 
> > Did you consider shadow stacks? IIRC we currently have userspace shadow
> > stack support available, and that will utterly break all of this.
> 
> nope.. I guess it's the extra ret instruction in the trampoline that would
> make it crash?

The original behavior seems problematic for shadow stack IIUC. I'm not sure of
the additional breakage with the new behavior.

Roughly, how shadow stack works is there is an additional protected stack for
the app thread. The HW pushes to from the shadow stack with CALL, and pops from
it with RET. But it also continues to push and pop from the normal stack. On
pop, if the values don't match between the two stacks, an exception is
generated. The whole point is to prevent the app from overwriting its stack
return address to return to random places.

Userspace cannot (normally) write to the shadow stack, but the kernel can do
this or adust the SSP (shadow stack pointer). So in the kernel (for things like
sigreturn) there is an ability to do what is needed. Ptracers also can do things
like this.