From: Andrey Grodzovsky <andrey.grodzovsky@crowdstrike.com>
To: Song Liu <song@kernel.org>
Cc: Petr Mladek <pmladek@suse.com>,
"kernel-team@lists.ubuntu.com" <kernel-team@lists.ubuntu.com>,
"live-patching@vger.kernel.org" <live-patching@vger.kernel.org>,
Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [External] Re: Question - Livepatch/Kprobe Coexistence on Ftrace-enabled Functions (Ubuntu kernel based on Linux stable 5.15.30)
Date: Fri, 17 Oct 2025 15:48:16 -0400 [thread overview]
Message-ID: <69339fb8-04a6-4c28-bb71-d9522ebd7282@crowdstrike.com> (raw)
In-Reply-To: <CAHzjS_sQQaTZpxC2drGx8=7zCMAKQN_CNjYFcNzxZEGhd+yXPA@mail.gmail.com>
On 10/17/25 15:07, Song Liu wrote:
> On Fri, Oct 17, 2025 at 9:58 AM Song Liu <song@kernel.org> wrote:
>> On Thu, Oct 16, 2025 at 2:55 PM Andrey Grodzovsky
>> <andrey.grodzovsky@crowdstrike.com> wrote:
>> [...]
>>> [AG] - Trying first to point him at the original function - but he
>>> fails on the fexit I assume - which is strange, I assumed fexit
>>> (kretfunc) and livepatch can coexist ?
>>>
>>> ubuntu@ip-10-10-114-204:~$ sudo bpftrace -e
>>> 'fentry:vmlinux:begin_new_exec { @start[tid] = nsecs; printf("-> EXEC
>>> START (fentry): PID %d, Comm %s\n", pid, comm); }
>>> fexit:vmlinux:begin_new_exec { $latency = nsecs - @start[tid];
>>> delete(@start[tid]); printf("<- EXEC END (fexit): PID %d, Comm %s,
>>> Retval %d, Latency %d us\n", pid, comm, retval, $latency / 1000); }'
>>> Attaching 2 probes...
>>> ERROR: Error attaching probe: kretfunc:vmlinux:begin_new_exec
>>>
>>> [AG] - Trying to skip the fexit and only do fentry - he still rejects
>>> ubuntu@ip-10-10-114-204:~$ sudo bpftrace -vvv -e
>>> 'fentry:vmlinux:begin_new_exec { @start[tid] = nsecs; printf("-> EXEC
>>> START (fentry): PID %d, Comm %s\n", pid, comm); }'
>>> sudo: unable to resolve host ip-10-10-114-204: Temporary failure in name
>>> resolution
>>> INFO: node count: 12
>>> Attaching 1 probe...
>>>
>>> Program ID: 295
>>>
>>> The verifier log:
>>> processed 50 insns (limit 1000000) max_states_per_insn 0 total_states 3
>>> peak_states 3 mark_read 1
>>>
>>> Attaching kfunc:vmlinux:begin_new_exec
>>> ERROR: Error attaching probe: kfunc:vmlinux:begin_new_exec
>> OK, I could reproduce this issue and found the issue. In my test,
>> fexit+livepatch works on some older kernel, but fails on some newer
>> kernel. I haven't bisected to the commit that broke it.
>>
>> Something like the following make it work:
>>
>> diff --git i/kernel/trace/ftrace.c w/kernel/trace/ftrace.c
>> index 2e113f8b13a2..4277b4f33eb8 100644
>> --- i/kernel/trace/ftrace.c
>> +++ w/kernel/trace/ftrace.c
>> @@ -5985,6 +5985,8 @@ int register_ftrace_direct(struct ftrace_ops
>> *ops, unsigned long addr)
>> ops->direct_call = addr;
>>
>> err = register_ftrace_function_nolock(ops);
>> + if (err)
>> + remove_direct_functions_hash(direct_functions, addr);
>>
>> out_unlock:
>> mutex_unlock(&direct_mutex);
>>
>> Andrey, could you also test this change?
> Attached is a better version of the fix.
>
> Thanks,
> Song
Thank you Song!
So, with this You say both fentry and fexit will work no issues ?
So juts to understand, as i am not familiar with live-patch generation,
I get the sources for my Ubuntu kernel, I apply your patch, I also
generate manually livepatch module that makes a dummy patching to my
test function (begin_new_exec), and apply this patch to my running
costum kernel ? Because i don't think the stadard ubuntu livepatching
will agree to apply his livepatch CVEs to my ostum kenel, it will
probably recognize it's not stock ubuntu kernel but amanully built one.
Thanks,
Andrey
So i need to build the stable
next prev parent reply other threads:[~2025-10-17 19:48 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-10-14 21:37 Question - Livepatch/Kprobe Coexistence on Ftrace-enabled Functions (Ubuntu kernel based on Linux stable 5.15.30) Andrey Grodzovsky
2025-10-15 11:53 ` Petr Mladek
2025-10-15 21:11 ` [External] " Andrey Grodzovsky
2025-10-16 10:56 ` Petr Mladek
2025-10-16 21:32 ` Song Liu
2025-10-16 21:55 ` Andrey Grodzovsky
2025-10-17 16:58 ` Song Liu
2025-10-17 19:07 ` Song Liu
2025-10-17 19:48 ` Andrey Grodzovsky [this message]
2025-10-17 21:11 ` Song Liu
2025-10-20 14:56 ` Andrey Grodzovsky
2025-10-20 16:03 ` Song Liu
2025-10-20 16:45 ` Andrey Grodzovsky
2025-10-20 18:53 ` Song Liu
2025-10-20 19:10 ` Andrey Grodzovsky
2025-10-20 19:44 ` Song Liu
2025-10-20 19:53 ` Andrey Grodzovsky
2025-10-20 21:31 ` Andrey Grodzovsky
2025-10-21 6:07 ` Song Liu
2025-10-21 14:09 ` Steven Rostedt
2025-10-21 16:06 ` Song Liu
2025-10-21 14:15 ` Andrey Grodzovsky
2025-10-21 16:08 ` Song Liu
2025-10-20 20:31 ` Steven Rostedt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=69339fb8-04a6-4c28-bb71-d9522ebd7282@crowdstrike.com \
--to=andrey.grodzovsky@crowdstrike.com \
--cc=kernel-team@lists.ubuntu.com \
--cc=live-patching@vger.kernel.org \
--cc=pmladek@suse.com \
--cc=rostedt@goodmis.org \
--cc=song@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox