Re: [External] Re: Question - Livepatch/Kprobe Coexistence on Ftrace-enabled Functions (Ubuntu kernel based on Linux stable 5.15.30)

[Andrey Grodzovsky <andrey.grodzovsky@crowdstrike.com>]

On 10/15/25 07:53, Petr Mladek wrote:
> On Tue 2025-10-14 21:37:49, Andrey Grodzovsky wrote:
>> Dear Upstream Livepatch  team and Ubuntu Kernel team - I included you both in this since the issue lies on the boundary between Ubuntu kernel and upstream.
>>
>> According to official kernel documentation  - https://urldefense.com/v3/__https://docs.kernel.org/livepatch/livepatch.html*livepatch__;Iw!!BmdzS3_lV9HdKG8!z3Y4vlE7RcCriT3z4Hg7cVaojZPN-ysQTbjDJVXyO_MoRRlkKsymUTDP4PGvvPaV0TDVYhziOYMm9WnUGu5TeFxUxQ$ , section 7, Limitations -
>> 1 - Kretprobes using the ftrace framework conflict with the patched functions.
>> 2 - Kprobes in the original function are ignored when the code is redirected to the new implementation.
>>
>> Yet, when testing on my Ubuntu 5.15.0-1005.7-aws (based on 5.15.30 stable kernel) machine, I have no problem applying Livepatch and then setting krpobes and kretprobes on a patched function using bpftrace (or directly by coding a BPF program with kprobe/kretprobe attachment)and can confirm both execute without issues. Also the opposite works fine, running my krpobe and kretprobe hooks doesn't prevent from livepatch to be applied successfully.
>>
>> fentry/fexit probes do fail in in both directions - but this is expected according to my understanding as coexistence of livepatching and ftrace based BPF hooks are mutually exclusive until 6.0 based kernels
>>
>> libbpf: prog 'begin_new_exec_fentry': failed to attach: Device or resource busy
>> libbpf: prog 'begin_new_exec_fentry': failed to auto-attach: -16
>>
>>
>> Please help me understand this contradiction about kprobes - is this because the KPROBES are FTRACE based  or any other reason ?
> Heh, it seems that we have discussed this 10 years ago and I already
> forgot most details.
>
> Yes, the conflict is detected when KPROBES are using FTRACE
> infrastructure. But it happens only when the KPROBE needs to redirect
> the function call, namely when it needs to modify IP address which will be used
> when all attached ftrace callbacks are proceed.
>
> It is related to the FTRACE_OPS_FL_IPMODIFY flag.


I see, that explains my case as my probes are simple, print only probes 
that defently don't that the ip pointer.

But i still don't understand limitation 2 above doesn't show itself - 
how my kprobes and especially kretprobes, continue execute even after 
livepatch applied to the function  i am attached to ? The code execution 
is redirected to a different function then to which i attached my probes...

Also - can you please confirm that as far as i checked, starting with 
kernel 6.0 fentry/fexit on x86 should not have any conflicts with 
livepatch per merge of this RFC - 
https://lkml.iu.edu/hypermail/linux/kernel/2207.2/00858.html ?

Thanks,
Andrey

>
> More details can be found in the discussion at
> https://urldefense.com/v3/__https://lore.kernel.org/all/20141121102516.11844.27829.stgit@localhost.localdomain/T/*re746846b6b16c49a55c89b4c63b7995fe5972111__;Iw!!BmdzS3_lV9HdKG8!z3Y4vlE7RcCriT3z4Hg7cVaojZPN-ysQTbjDJVXyO_MoRRlkKsymUTDP4PGvvPaV0TDVYhziOYMm9WnUGu6pjuIgig$
>
> I seems that I made some analyze when it worked and it did not work,
> see https://urldefense.com/v3/__https://lore.kernel.org/all/20141121102516.11844.27829.stgit@localhost.localdomain/T/*mffd8c8bf4325b473d89876f2805f42f1af7c82d7__;Iw!!BmdzS3_lV9HdKG8!z3Y4vlE7RcCriT3z4Hg7cVaojZPN-ysQTbjDJVXyO_MoRRlkKsymUTDP4PGvvPaV0TDVYhziOYMm9WnUGu5xbeoulA$
> But I am not 100% sure that it was correct. Also it was before the
> BPF-boom.
>
> Also you might look at the selftest in the todays Linus' tree:
>
>    + tools/testing/selftests/livepatch/https://urldefense.com/v3/__http://test-kprobe.sh__;!!BmdzS3_lV9HdKG8!z3Y4vlE7RcCriT3z4Hg7cVaojZPN-ysQTbjDJVXyO_MoRRlkKsymUTDP4PGvvPaV0TDVYhziOYMm9WnUGu5RXF-AnA$
>    + tools/testing/selftests/livepatch/test_modules/test_klp_kprobe.c
>    + tools/testing/selftests/livepatch/test_modules/test_klp_livepatch.c
>
> The parallel load fails when the Kprobe is using a post_handler.
>
> Sigh, we should fix the livepatch documentation. The kretprobes
> obviously work. register_kretprobe() even explicitely sets:
>
> 	rp->kp.post_handler = NULL;
>
> It seems that .post_handler is called after all ftrace handlers.
> And it sets IP after the NOPs, see kprobe_ftrace_handler().
> I am not sure about the use case.
>
> Best Regards,
> Petr