Re: [PATCH] kernel/trace/ftrace: introduce ftrace module notifier

Re: [PATCH] kernel/trace/ftrace: introduce ftrace module notifier

[Miroslav Benes]

[-- Attachment #1: Type: text/plain, Size: 3620 bytes --]

Hi,

+ Cc: live-patching@vger.kernel.org

On Thu, 26 Feb 2026, Song Chen wrote:

> Hi,
> 
> 在 2026/2/26 08:27, Steven Rostedt 写道:
> > On Wed, 25 Feb 2026 13:46:39 +0800
> > chensong_2000@189.cn wrote:
> > 
> >> From: Song Chen <chensong_2000@189.cn>
> >>
> >> Like kprobe, fprobe and btf, this patch attempts to introduce
> >> a notifier_block for ftrace to decouple its initialization from
> >> load_module.
> >>
> >> Below is the table of ftrace fucntions calls in different
> >> module state:
> >>
> >>  MODULE_STATE_UNFORMED	ftrace_module_init
> >>  MODULE_STATE_COMING	ftrace_module_enable
> >>  MODULE_STATE_LIVE	ftrace_free_mem
> >>  MODULE_STATE_GOING	ftrace_release_mod
> >>
> >> Unlike others, ftrace module notifier must take care of state
> >> MODULE_STATE_UNFORMED to ensure calling ftrace_module_init
> >> before complete_formation which changes module's text property.
> >>
> >> That pretty much remains same logic with its original design,
> >> the only thing that changes is blocking_notifier_call_chain
> >> (MODULE_STATE_GOING) has to be moved from coming_cleanup to
> >> ddebug_cleanup in function load_module to ensure
> >> ftrace_release_mod is invoked in case complete_formation fails.
> >>
> >> Signed-off-by: Song Chen <chensong_2000@189.cn>
> >> ---
> >>   kernel/module/main.c  | 14 ++++----------
> >>   kernel/trace/ftrace.c | 37 +++++++++++++++++++++++++++++++++++++
> >>   2 files changed, 41 insertions(+), 10 deletions(-)
> >>
> >> diff --git a/kernel/module/main.c b/kernel/module/main.c
> >> index 710ee30b3bea..5dc0a980e9bd 100644
> >> --- a/kernel/module/main.c
> >> +++ b/kernel/module/main.c
> >> @@ -45,7 +45,6 @@
> >>   #include <linux/license.h>
> >>   #include <asm/sections.h>
> >>   #include <linux/tracepoint.h>
> >> -#include <linux/ftrace.h>
> >>   #include <linux/livepatch.h>
> >>   #include <linux/async.h>
> >>   #include <linux/percpu.h>
> >> @@ -836,7 +835,6 @@ SYSCALL_DEFINE2(delete_module, const char __user *,
> >> name_user,
> >>    blocking_notifier_call_chain(&module_notify_list,
> >>    			     MODULE_STATE_GOING, mod);
> >>   	klp_module_going(mod);
> >> -	ftrace_release_mod(mod);
> > 
> > Is the above safe? klp uses ftrace. That means klp_module_going() may
> > need to be called before ftrace_release_mod(). That said, I wonder if
> > klp_module_going() could be moved into ftrace_release_mod()?
> > 
> >>   
> 
> I didn't test with klp, so i'm not sure if it's safe. But i consider klp is
> the other part which should be decoupled after ftrace and klp should introduce
> its own notifier.
> 
> If klp_module_going must be running before ftrace_release_mod, i can try to
> use priority in notifier_block to ensure their order.
> 
> Let me see if there is any way to use notifier and remain below calling
> sequence:
> 
> ftrace_module_enable
> klp_module_coming
> blocking_notifier_call_chain_robust(MODULE_STATE_COMING)
> 
> blocking_notifier_call_chain(MODULE_STATE_GOING)
> klp_module_going
> ftrace_release_mod

Both klp and ftrace used module notifiers in the past. We abandoned that 
and opted for direct calls due to issues with ordering at the time. I do 
not have the list of problems at hand but I remember it was very fragile.

See commits 7dcd182bec27 ("ftrace/module: remove ftrace module 
notifier"), 7e545d6eca20 ("livepatch/module: remove livepatch module 
notifier") and their surroundings.

So unless there is a reason for the change (which should be then carefully 
reviewed and properly tested), I would prefer to keep it as is. What is 
the motivation? I am failing to find it in the commit log.

Regards,
Miroslav

Re: [PATCH] kernel/trace/ftrace: introduce ftrace module notifier

[Steven Rostedt]

On Thu, 26 Feb 2026 11:51:53 +0100 (CET)
Miroslav Benes <mbenes@suse.cz> wrote:

> > Let me see if there is any way to use notifier and remain below calling
> > sequence:
> > 
> > ftrace_module_enable
> > klp_module_coming
> > blocking_notifier_call_chain_robust(MODULE_STATE_COMING)
> > 
> > blocking_notifier_call_chain(MODULE_STATE_GOING)
> > klp_module_going
> > ftrace_release_mod  
> 
> Both klp and ftrace used module notifiers in the past. We abandoned that 
> and opted for direct calls due to issues with ordering at the time. I do 
> not have the list of problems at hand but I remember it was very fragile.
> 
> See commits 7dcd182bec27 ("ftrace/module: remove ftrace module 
> notifier"), 7e545d6eca20 ("livepatch/module: remove livepatch module 
> notifier") and their surroundings.
> 
> So unless there is a reason for the change (which should be then carefully 
> reviewed and properly tested), I would prefer to keep it as is. What is 
> the motivation? I am failing to find it in the commit log.

Honestly, I do think just decoupling ftrace and live kernel patching from
modules is rationale enough, as it makes the code a bit cleaner. But to do
so, we really need to make sure there is absolutely no regressions.

Thus, to allow such a change, I would ask those that are proposing it, show
a full work flow of how ftrace, live kernel patching, and modules work with
each other and why those functions are currently injected in the module code.

As Miroslav stated, we tried to do it via notifiers in the past and it
failed. I don't want to find out why they failed by just adding them back
to notifiers again. Instead, the reasons must be fully understood and
updates made to make sure they will not fail in the future.

-- Steve

Re: [PATCH] kernel/trace/ftrace: introduce ftrace module notifier

[Song Chen]

Hi,

在 2026/2/27 01:30, Steven Rostedt 写道:
> On Thu, 26 Feb 2026 11:51:53 +0100 (CET)
> Miroslav Benes <mbenes@suse.cz> wrote:
> 
>>> Let me see if there is any way to use notifier and remain below calling
>>> sequence:
>>>
>>> ftrace_module_enable
>>> klp_module_coming
>>> blocking_notifier_call_chain_robust(MODULE_STATE_COMING)
>>>
>>> blocking_notifier_call_chain(MODULE_STATE_GOING)
>>> klp_module_going
>>> ftrace_release_mod
>>
>> Both klp and ftrace used module notifiers in the past. We abandoned that
>> and opted for direct calls due to issues with ordering at the time. I do
>> not have the list of problems at hand but I remember it was very fragile.
>>
>> See commits 7dcd182bec27 ("ftrace/module: remove ftrace module
>> notifier"), 7e545d6eca20 ("livepatch/module: remove livepatch module
>> notifier") and their surroundings.
>>
>> So unless there is a reason for the change (which should be then carefully
>> reviewed and properly tested), I would prefer to keep it as is. What is
>> the motivation? I am failing to find it in the commit log.

There is no special motivation, i just read btf initialization in module 
loading and found direct calls of ftrace and klp, i thought they were 
just forgotten to use notifier and i even didn't search git log to 
verify, sorry about that.

> 
> Honestly, I do think just decoupling ftrace and live kernel patching from
> modules is rationale enough, as it makes the code a bit cleaner. But to do
> so, we really need to make sure there is absolutely no regressions.
> 
> Thus, to allow such a change, I would ask those that are proposing it, show
> a full work flow of how ftrace, live kernel patching, and modules work with
> each other and why those functions are currently injected in the module code.
> 
> As Miroslav stated, we tried to do it via notifiers in the past and it
> failed. I don't want to find out why they failed by just adding them back
> to notifiers again. Instead, the reasons must be fully understood and
> updates made to make sure they will not fail in the future.

Yes, you are right, i read commit msg of 7dcd182bec27, this patch just 
reverses it simply and will introduce order issue back. I will try to 
find out the problem in the past at first.

Thank you both.

/Song

> 
> -- Steve
> 
> 

Re: [PATCH] kernel/trace/ftrace: introduce ftrace module notifier

[Petr Mladek]

On Fri 2026-02-27 09:34:59, Song Chen wrote:
> Hi,
> 
> 在 2026/2/27 01:30, Steven Rostedt 写道:
> > On Thu, 26 Feb 2026 11:51:53 +0100 (CET)
> > Miroslav Benes <mbenes@suse.cz> wrote:
> > 
> > > > Let me see if there is any way to use notifier and remain below calling
> > > > sequence:
> > > > 
> > > > ftrace_module_enable
> > > > klp_module_coming
> > > > blocking_notifier_call_chain_robust(MODULE_STATE_COMING)
> > > > 
> > > > blocking_notifier_call_chain(MODULE_STATE_GOING)
> > > > klp_module_going
> > > > ftrace_release_mod
> > > 
> > > Both klp and ftrace used module notifiers in the past. We abandoned that
> > > and opted for direct calls due to issues with ordering at the time. I do
> > > not have the list of problems at hand but I remember it was very fragile.
> > > 
> > > See commits 7dcd182bec27 ("ftrace/module: remove ftrace module
> > > notifier"), 7e545d6eca20 ("livepatch/module: remove livepatch module
> > > notifier") and their surroundings.
> > > 
> > > So unless there is a reason for the change (which should be then carefully
> > > reviewed and properly tested), I would prefer to keep it as is. What is
> > > the motivation? I am failing to find it in the commit log.
> 
> There is no special motivation, i just read btf initialization in module
> loading and found direct calls of ftrace and klp, i thought they were just
> forgotten to use notifier and i even didn't search git log to verify, sorry
> about that.
> 
> > 
> > Honestly, I do think just decoupling ftrace and live kernel patching from
> > modules is rationale enough, as it makes the code a bit cleaner. But to do
> > so, we really need to make sure there is absolutely no regressions.
> > 
> > Thus, to allow such a change, I would ask those that are proposing it, show
> > a full work flow of how ftrace, live kernel patching, and modules work with
> > each other and why those functions are currently injected in the module code.
> > 
> > As Miroslav stated, we tried to do it via notifiers in the past and it
> > failed. I don't want to find out why they failed by just adding them back
> > to notifiers again. Instead, the reasons must be fully understood and
> > updates made to make sure they will not fail in the future.
> 
> Yes, you are right, i read commit msg of 7dcd182bec27, this patch just
> reverses it simply and will introduce order issue back. I will try to find
> out the problem in the past at first.

AFAIK, the root of the problem is that livepatch uses the ftrace
framework. It means that:

   + ftrace must be initialized before livepatch gets enabled
   + livepatch must be disabled before ftrace support gets removed

My understanding is that this can't be achieved by notifiers easily
because they are always proceed in the same order.

An elegant solution would be to introduce  notifier_reverse_call_chain()
which would process the callbacks in the reverse order. But it might
be non-trivial:

  + We would need to make sure that it does not break some
    existing "hidden" dependencies.

  + notifier_call_chain() uses RCU to process the list of registered
    callbacks. I am not sure how complicated would be to make it safe
    in both directions.

Best Regards,
Petr