Live Patching
 help / color / mirror / Atom feed
From: Andrey Grodzovsky <andrey.grodzovsky@crowdstrike.com>
To: Song Liu <song@kernel.org>
Cc: Petr Mladek <pmladek@suse.com>,
	"kernel-team@lists.ubuntu.com" <kernel-team@lists.ubuntu.com>,
	"live-patching@vger.kernel.org" <live-patching@vger.kernel.org>,
	Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [External] Re: Question - Livepatch/Kprobe Coexistence on Ftrace-enabled Functions (Ubuntu kernel based on Linux stable 5.15.30)
Date: Fri, 17 Oct 2025 15:48:16 -0400	[thread overview]
Message-ID: <69339fb8-04a6-4c28-bb71-d9522ebd7282@crowdstrike.com> (raw)
In-Reply-To: <CAHzjS_sQQaTZpxC2drGx8=7zCMAKQN_CNjYFcNzxZEGhd+yXPA@mail.gmail.com>

On 10/17/25 15:07, Song Liu wrote:
> On Fri, Oct 17, 2025 at 9:58 AM Song Liu <song@kernel.org> wrote:
>> On Thu, Oct 16, 2025 at 2:55 PM Andrey Grodzovsky
>> <andrey.grodzovsky@crowdstrike.com> wrote:
>> [...]
>>> [AG] - Trying first to point him at the original  function - but he
>>> fails on the fexit I assume  - which is strange, I assumed fexit
>>> (kretfunc) and livepatch can coexist ?
>>>
>>> ubuntu@ip-10-10-114-204:~$ sudo bpftrace -e
>>> 'fentry:vmlinux:begin_new_exec { @start[tid] = nsecs; printf("-> EXEC
>>> START (fentry): PID %d, Comm %s\n", pid, comm); }
>>> fexit:vmlinux:begin_new_exec { $latency = nsecs - @start[tid];
>>> delete(@start[tid]); printf("<- EXEC END (fexit): PID %d, Comm %s,
>>> Retval %d, Latency %d us\n", pid, comm, retval, $latency / 1000); }'
>>> Attaching 2 probes...
>>> ERROR: Error attaching probe: kretfunc:vmlinux:begin_new_exec
>>>
>>> [AG] - Trying to skip the fexit and only do fentry - he still rejects
>>> ubuntu@ip-10-10-114-204:~$ sudo bpftrace -vvv -e
>>> 'fentry:vmlinux:begin_new_exec { @start[tid] = nsecs; printf("-> EXEC
>>> START (fentry): PID %d, Comm %s\n", pid, comm); }'
>>> sudo: unable to resolve host ip-10-10-114-204: Temporary failure in name
>>> resolution
>>> INFO: node count: 12
>>> Attaching 1 probe...
>>>
>>> Program ID: 295
>>>
>>> The verifier log:
>>> processed 50 insns (limit 1000000) max_states_per_insn 0 total_states 3
>>> peak_states 3 mark_read 1
>>>
>>> Attaching kfunc:vmlinux:begin_new_exec
>>> ERROR: Error attaching probe: kfunc:vmlinux:begin_new_exec
>> OK, I could reproduce this issue and found the issue. In my test,
>> fexit+livepatch works on some older kernel, but fails on some newer
>> kernel. I haven't bisected to the commit that broke it.
>>
>> Something like the following make it work:
>>
>> diff --git i/kernel/trace/ftrace.c w/kernel/trace/ftrace.c
>> index 2e113f8b13a2..4277b4f33eb8 100644
>> --- i/kernel/trace/ftrace.c
>> +++ w/kernel/trace/ftrace.c
>> @@ -5985,6 +5985,8 @@ int register_ftrace_direct(struct ftrace_ops
>> *ops, unsigned long addr)
>>          ops->direct_call = addr;
>>
>>          err = register_ftrace_function_nolock(ops);
>> +       if (err)
>> +               remove_direct_functions_hash(direct_functions, addr);
>>
>>    out_unlock:
>>          mutex_unlock(&direct_mutex);
>>
>> Andrey, could you also test this change?
> Attached is a better version of the fix.
>
> Thanks,
> Song

Thank you Song!

So, with this You say both fentry and fexit will work no issues ?

So juts to understand, as i am not familiar with live-patch generation, 
I get the sources for my Ubuntu kernel, I apply your patch, I also 
generate manually livepatch module that makes a dummy patching to my 
test function (begin_new_exec), and apply this patch to my running 
costum kernel ? Because i don't think the stadard ubuntu livepatching 
will agree to apply his livepatch CVEs to my ostum kenel, it will 
probably recognize it's not stock ubuntu kernel but amanully built one.

Thanks,
Andrey

So i need to build the stable


  reply	other threads:[~2025-10-17 19:48 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-10-14 21:37 Question - Livepatch/Kprobe Coexistence on Ftrace-enabled Functions (Ubuntu kernel based on Linux stable 5.15.30) Andrey Grodzovsky
2025-10-15 11:53 ` Petr Mladek
2025-10-15 21:11   ` [External] " Andrey Grodzovsky
2025-10-16 10:56     ` Petr Mladek
2025-10-16 21:32       ` Song Liu
2025-10-16 21:55         ` Andrey Grodzovsky
2025-10-17 16:58           ` Song Liu
2025-10-17 19:07             ` Song Liu
2025-10-17 19:48               ` Andrey Grodzovsky [this message]
2025-10-17 21:11                 ` Song Liu
2025-10-20 14:56                   ` Andrey Grodzovsky
2025-10-20 16:03                     ` Song Liu
2025-10-20 16:45                       ` Andrey Grodzovsky
2025-10-20 18:53                         ` Song Liu
2025-10-20 19:10                           ` Andrey Grodzovsky
2025-10-20 19:44                             ` Song Liu
2025-10-20 19:53                             ` Andrey Grodzovsky
2025-10-20 21:31                               ` Andrey Grodzovsky
2025-10-21  6:07                                 ` Song Liu
2025-10-21 14:09                                   ` Steven Rostedt
2025-10-21 16:06                                     ` Song Liu
2025-10-21 14:15                                   ` Andrey Grodzovsky
2025-10-21 16:08                                     ` Song Liu
2025-10-20 20:31                           ` Steven Rostedt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=69339fb8-04a6-4c28-bb71-d9522ebd7282@crowdstrike.com \
    --to=andrey.grodzovsky@crowdstrike.com \
    --cc=kernel-team@lists.ubuntu.com \
    --cc=live-patching@vger.kernel.org \
    --cc=pmladek@suse.com \
    --cc=rostedt@goodmis.org \
    --cc=song@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox