[LSF/MM/BPF TOPIC] BPF deferred stack trace unwinder

[Steven Rostedt <rostedt@goodmis.org>]

Hopefully this isn't too late but I got side tracked and never submitted.

We are currently working on enabling SFrames[1] to get user space stack
traces. This would allow us to retrieve reliable user space stack
traces from inside the kernel without the need for frame pointers. The
issue with SFrames is that they require reading user space memory. That
means reading the SFrame data can not be performed from an interrupt
context as the read will likely cause a major page fault.

To handle this, a deferred user stack trace unwinder was added to the
kernel[2]. How this works is to do the unwinding where it is safe to
have major page faults. The obvious place for that is just before going
back to user space (via task work). A tracer (like BPF) would call
unwind_deferred_init() to register itself with the unwinder and give it
a callback that gets called  before the task goes back to user space
with the user stack trace as well as a "cookie". BPF would call
unwind_deferred_request() at the time it wants the user space
stacktrace (could be in any context, like an interrupt or even an NMI).
That function returns a unique "cookie" that represents the user space
stack trace its callback will receive (along with the same cookie as a
parameter). Then BPF could record this cookie and have user space
mapping tie it together with whatever else was recorded then (like the
kernel stack trace).

Blake Jones brought up an issue that their tooling has with this
approach. That is it may be difficult to keep track of all the kernel
stack traces it needs to map to the user space stack trace. The reason
this is a problem is because the tooling saves the kernel and user
stack trace into a special hash. It is performed after a task is
scheduled out and back in and it records the time the task was off the
CPU with each kernel/user stack trace.

Currently the recording is done when the task schedules back in. But
due to faulting, it is not safe to call sframes unwinding at a moment
the task schedules in. It must wait until the task goes back to user
space and then tie all the kernel tracer to where the task scheduled
out with the user space stacktrace to create the hash and save it. As a
system call may have a hundred different places it can schedule out,
the tool can't cache all the kernel stack traces waiting for it to
schedule back in, as that would require saving a hundred stack traces
for every task. You can read more about the issue[3].

I would like to bring this topic up at LSF/MM/BPF.

-- Steve


[1] https://sourceware.org/binutils/docs/sframe-spec.html
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c6439bfaabf25b736154ac5640c677da2c085db4
[3] https://lore.kernel.org/all/20260126142118.2ea3cf13@gandalf.local.home/