Re: [External] Re: Question - Livepatch/Kprobe Coexistence on Ftrace-enabled Functions (Ubuntu kernel based on Linux stable 5.15.30)

[Andrey Grodzovsky <andrey.grodzovsky@crowdstrike.com>]

On 10/17/25 17:11, Song Liu wrote:
> On Fri, Oct 17, 2025 at 12:48 PM Andrey Grodzovsky
> <andrey.grodzovsky@crowdstrike.com> wrote:
>> On 10/17/25 15:07, Song Liu wrote:
>>> On Fri, Oct 17, 2025 at 9:58 AM Song Liu <song@kernel.org> wrote:
>>>> On Thu, Oct 16, 2025 at 2:55 PM Andrey Grodzovsky
>>>> <andrey.grodzovsky@crowdstrike.com> wrote:
>>>> [...]
>>>>> [AG] - Trying first to point him at the original  function - but he
>>>>> fails on the fexit I assume  - which is strange, I assumed fexit
>>>>> (kretfunc) and livepatch can coexist ?
>>>>>
>>>>> ubuntu@ip-10-10-114-204:~$ sudo bpftrace -e
>>>>> 'fentry:vmlinux:begin_new_exec { @start[tid] = nsecs; printf("-> EXEC
>>>>> START (fentry): PID %d, Comm %s\n", pid, comm); }
>>>>> fexit:vmlinux:begin_new_exec { $latency = nsecs - @start[tid];
>>>>> delete(@start[tid]); printf("<- EXEC END (fexit): PID %d, Comm %s,
>>>>> Retval %d, Latency %d us\n", pid, comm, retval, $latency / 1000); }'
>>>>> Attaching 2 probes...
>>>>> ERROR: Error attaching probe: kretfunc:vmlinux:begin_new_exec
>>>>>
>>>>> [AG] - Trying to skip the fexit and only do fentry - he still rejects
>>>>> ubuntu@ip-10-10-114-204:~$ sudo bpftrace -vvv -e
>>>>> 'fentry:vmlinux:begin_new_exec { @start[tid] = nsecs; printf("-> EXEC
>>>>> START (fentry): PID %d, Comm %s\n", pid, comm); }'
>>>>> sudo: unable to resolve host ip-10-10-114-204: Temporary failure in name
>>>>> resolution
>>>>> INFO: node count: 12
>>>>> Attaching 1 probe...
>>>>>
>>>>> Program ID: 295
>>>>>
>>>>> The verifier log:
>>>>> processed 50 insns (limit 1000000) max_states_per_insn 0 total_states 3
>>>>> peak_states 3 mark_read 1
>>>>>
>>>>> Attaching kfunc:vmlinux:begin_new_exec
>>>>> ERROR: Error attaching probe: kfunc:vmlinux:begin_new_exec
>>>> OK, I could reproduce this issue and found the issue. In my test,
>>>> fexit+livepatch works on some older kernel, but fails on some newer
>>>> kernel. I haven't bisected to the commit that broke it.
>>>>
>>>> Something like the following make it work:
>>>>
>>>> diff --git i/kernel/trace/ftrace.c w/kernel/trace/ftrace.c
>>>> index 2e113f8b13a2..4277b4f33eb8 100644
>>>> --- i/kernel/trace/ftrace.c
>>>> +++ w/kernel/trace/ftrace.c
>>>> @@ -5985,6 +5985,8 @@ int register_ftrace_direct(struct ftrace_ops
>>>> *ops, unsigned long addr)
>>>>           ops->direct_call = addr;
>>>>
>>>>           err = register_ftrace_function_nolock(ops);
>>>> +       if (err)
>>>> +               remove_direct_functions_hash(direct_functions, addr);
>>>>
>>>>     out_unlock:
>>>>           mutex_unlock(&direct_mutex);
>>>>
>>>> Andrey, could you also test this change?
>>> Attached is a better version of the fix.
>>>
>>> Thanks,
>>> Song
>> Thank you Song!
>>
>> So, with this You say both fentry and fexit will work no issues ?
> Yes, fentry and fexit should both work.
>
>> So juts to understand, as i am not familiar with live-patch generation,
>> I get the sources for my Ubuntu kernel, I apply your patch, I also
>> generate manually livepatch module that makes a dummy patching to my
>> test function (begin_new_exec), and apply this patch to my running
>> costum kernel ? Because i don't think the stadard ubuntu livepatching
>> will agree to apply his livepatch CVEs to my ostum kenel, it will
>> probably recognize it's not stock ubuntu kernel but amanully built one.
> livepatch is a kernel module. Therefore, unless the two kernels are almost
> identical, livepatch built for one kernel cannot be used on the other.
>
> If you build the kernel from source code, there are some samples in
> samples/livepatch that you can use for testing. PS: You need to enable
>
>    CONFIG_SAMPLE_LIVEPATCH=m
>
> I hope this helps.

Thanks Song, working on repro, kernel rebuilt, test module is loading 
but, bpftrace is refusing to attach now to fentries/fexits claiming the 
costum kernel is not supporting it. It did
attach in the case of stock AWS kernel i copied the .config from. So 
just trying to figure out now if some Kcofnig flags are missing or 
different . Let me know in case you manage to confirm yourself in the 
meanwhile the fix works for
you.

Thanks,
Andrey


>
> Thanks,
> Song