Re: [BUG?] ppc64le: fentry BPF not triggered after live patch (v6.14)

Naveen N Rao
2025-04-03T15:30:11Z

On Tue, Apr 01, 2025 at 02:48:10PM +0200, Jiri Olsa wrote:
> On Mon, Mar 31, 2025 at 10:09:40AM -0400, Steven Rostedt wrote:
> > On Mon, 31 Mar 2025 21:19:36 +0800
> > Shung-Hsi Yu <shung-hsi.yu@suse.com> wrote:
> >
> > > Hi all,
> > >
> > > On ppc64le (v6.14, kernel config attached), I've observed that fentry
> > > BPF programs stop being invoked after the target kernel function is live
> > > patched. This occurs regardless of whether the BPF program was attached
> > > before or after the live patch. I believe fentry/fprobe on ppc64le is
> > > added with [1].
> > >
> > > Steps to reproduce on ppc64le:
> > > - Use bpftrace (v0.10.0+) to attach a BPF program to cmdline_proc_show
> > > with fentry (kfunc is the older name bpftrace used for fentry, used
> > > here for max compatability)
> > >
> > > bpftrace -e 'kfunc:cmdline_proc_show { printf("%lld: cmdline_proc_show() called by %s\n", nsecs(), comm) }'
> > >
> > > - Run `cat /proc/cmdline` and observe bpftrace output
> > >
> > > - Load samples/livepatch/livepatch-sample.ko
> > >
> > > - Run `cat /proc/cmdline` again. Observe "this has been live patched" in
> > > output, but no new bpftrace output.
> > >
> > > Note: once the live patching module is disabled through the sysfs interface
> > > the BPF program invocation is restored.
> > >
> > > Is this the expected interaction between fentry BPF and live patching?
> > > On x86_64 it does _not_ happen, so I'd guess the behavior on ppc64le is
> > > unintended. Any insights appreciated.

We haven't addressed this particular interaction in the powerpc support
for ftrace direct and BPF trampolines. Right now, live patching takes
priority so we call the livepatch'ed function and skip further ftrace
direct calls.

I'm curious if this works on arm64 with which we share support for
DYNAMIC_FTRACE_WITH_CALL_OPS.

> >
> > Hmm, I'm not sure how well BPF function attachment and live patching
> > interact. Can you see if on x86 the live patch is actually updated when a
> > BPF program is attached?
>
> above works for me on x86, there's both 'this has been live patched'
> and bpftrace output
>
> >
> > Would be even more interesting to see how BPF reading the return code works
> > with live patching, as it calls the function directly from the BPF
> > trampoline. I wonder, does it call the live patched function, or does it
> > call the original one?
>
> yes, that should work, Song fixed some time ago with:
> 00963a2e75a8 bpf: Support bpf_trampoline on functions with IPMODIFY (e.g. livepatch)

We don't support BPF_TRAMP_F_ORIG_STACK today, and I am not sure that
will be easy to support with the out-of-line stubs we are using for
ftrace.

Today, we support one of livepatch or a direct caller from
ftrace_caller. Livepatch takes priority. We could call the direct caller
first, but that will only work if it is guaranteed to call the
livepatched function (i.e., if the direct call is to a bpf trampoline
with BPF_TRAMP_F_CALL_ORIG). Otherwise, we will need to call the
livepatched function after that, but we don't return to ftrace_caller
from the livepatch handler.

Not sure which of those interactions are valid between a livepatch and a
direct caller (with and without BPF_TRAMP_F_CALL_ORIG), and if all
combinations of those need to be supported.

- Naveen