From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
To: yebin <yebin@huaweicloud.com>
Cc: naveen@kernel.org, davem@davemloft.net,
linux-trace-kernel@vger.kernel.org, yebin10@huawei.com,
Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [PATCH] kprobes: avoid crash when rmmod/insmod modules after ftrace_disabled
Date: Tue, 2 Dec 2025 11:32:48 +0900 [thread overview]
Message-ID: <20251202113248.e9e2e7c665bd3ee07b83e399@kernel.org> (raw)
In-Reply-To: <6929089E.10706@huaweicloud.com>
Cc: Steve,
Since this is caused by ftrace_kill.
On Fri, 28 Nov 2025 10:27:42 +0800
yebin <yebin@huaweicloud.com> wrote:
>
>
> On 2025/11/27 12:18, Masami Hiramatsu (Google) wrote:
> > On Thu, 27 Nov 2025 12:52:48 +0900
> > Masami Hiramatsu (Google) <mhiramat@kernel.org> wrote:
> >
> >> Hi,
> >>
> >> Thanks for reporting!
> >>
> >>
> >> On Tue, 25 Nov 2025 10:05:36 +0800
> >> Ye Bin <yebin@huaweicloud.com> wrote:
> >>
> >>> From: Ye Bin <yebin10@huawei.com>
> >>>
> >>> There's a issue as follows when rmmod modules after ftrace disabled:
> >>
> >> You may see something like;
> >>
> >> Failed to unregister kprobe-ftrace (error -19)
> >>
> >> or
> >>
> >> Failed to disarm kprobe-ftrace at <function name> (error -19)
> >>
> >> right before this BUG, don't you?
> >> If you reported with that line, it's more easier to understand.
> >>
> Yes, there is indeed a warning generated. I might not have expressed it
> clearly enough. The issue below is related to the problem that occurs
> when the second module is unloaded. When the first module was unloaded,
> some nodes were left in the hash list, causing a use-after-free (UAF)
> issue when traversing the hash list.
> Therefore, this patch aims to resolve the UAF problem caused by residual
> nodes in the hash list after unloading a module while ftrace is disabled.
Yes, but I think your patch is redundant. Can you test the code
which I suggested at the last?
BTW, can you explain why that ftrace_disabled was kicked?
That usually means ftrace hits some bug.
> >>
> >>> BUG: unable to handle page fault for address: fffffbfff805000d
> >>> PGD 817fcc067 P4D 817fcc067 PUD 817fc8067 PMD 101555067 PTE 0
> >>> Oops: Oops: 0000 [#1] SMP KASAN PTI
> >>> CPU: 4 UID: 0 PID: 2012 Comm: rmmod Tainted: G W OE
> >>> Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
> >>> RIP: 0010:kprobes_module_callback+0x89/0x790
> >>> RSP: 0018:ffff88812e157d30 EFLAGS: 00010a02
> >>> RAX: 1ffffffff805000d RBX: dffffc0000000000 RCX: ffffffff86a8de90
> >>> RDX: ffffed1025c2af9b RSI: 0000000000000008 RDI: ffffffffc0280068
> >>> RBP: 0000000000000000 R08: 0000000000000001 R09: ffffed1025c2af9a
> >>> R10: ffff88812e157cd7 R11: 205d323130325420 R12: 0000000000000002
> >>> R13: ffffffffc0290488 R14: 0000000000000002 R15: ffffffffc0280040
> >>> FS: 00007fbc450dd740(0000) GS:ffff888420331000(0000) knlGS:0000000000000000
> >>> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> >>> CR2: fffffbfff805000d CR3: 000000010f624000 CR4: 00000000000006f0
> >>> Call Trace:
> >>> <TASK>
> >>> notifier_call_chain+0xc6/0x280
> >>> blocking_notifier_call_chain+0x60/0x90
> >>> __do_sys_delete_module.constprop.0+0x32a/0x4e0
> >>> do_syscall_64+0x5d/0xfa0
> >>> entry_SYSCALL_64_after_hwframe+0x76/0x7e
> >>>
> >>> The above issue occurs because the kprobe was not removed from the hash
> >>> list after ftrace_disable.
> >>> To prevent the system from restarting unexpectedly after ftrace_disable,
> >>> in such cases, unregister_kprobe() ensures that the probe is removed from
> >>> the hash list, preventing subsequent access to already freed memory.
> >>>
> >>> Fixes: 6f0f1dd71953 ("kprobes: Cleanup disabling and unregistering path")
> >>> Signed-off-by: Ye Bin <yebin10@huawei.com>
> >>> ---
> >>> kernel/kprobes.c | 26 ++++++++++++++++++++++++--
> >>> 1 file changed, 24 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/kernel/kprobes.c b/kernel/kprobes.c
> >>> index ab8f9fc1f0d1..d735a608b810 100644
> >>> --- a/kernel/kprobes.c
> >>> +++ b/kernel/kprobes.c
> >>> @@ -1731,8 +1731,30 @@ static int __unregister_kprobe_top(struct kprobe *p)
> >>>
> >>> /* Disable kprobe. This will disarm it if needed. */
> >>> ap = __disable_kprobe(p);
> >>> - if (IS_ERR(ap))
> >>> - return PTR_ERR(ap);
> >>> + if (IS_ERR(ap)) {
> >>> + int ret = PTR_ERR(ap);
> >>> +
> >>> + /*
> >>> + * If ftrace disabled we need to delete kprobe node from
> >>> + * hlist or aggregation list. If nodes are not removed when
> >>> + * modules are removed, the already released nodes will
> >>> + * remain in the linked list. Subsequent access to the
> >>> + * linked list may then trigger exceptions.
> >>> + */
> >>> + if (ret != -ENODEV)
> >>> + return ret;
> >>> +
> >>> + ap = __get_valid_kprobe(p);
> >>> + if (!ap)
> >>> + return ret;
> >>> +
> >>> + if (ap == p)
> >>> + hlist_del_rcu(&ap->hlist);
> >>> + else
> >>> + list_del_rcu(&p->list);
> >>
> >> Instead of repeating this process, we should ignore
> >> -ENODEV error from ftrace directly. BTW, ftrace_disabled is set
> >> when ftrace_kill() is called, that means ftrace is no more usable.
> >> So I think we can just ignore ftrace operation in
> >> __disarm_kprobe_ftrace().
> >
> > So, what we need is;
> >
> > diff --git a/kernel/kprobes.c b/kernel/kprobes.c
> > index ab8f9fc1f0d1..17d451553389 100644
> > --- a/kernel/kprobes.c
> > +++ b/kernel/kprobes.c
> > @@ -1104,6 +1104,10 @@ static int __disarm_kprobe_ftrace(struct kprobe *p, struct ftrace_ops *ops,
> > int ret;
> >
> > lockdep_assert_held(&kprobe_mutex);
> > + if (unlikely(kprobe_ftrace_disabled)) {
> > + /* Now ftrace is disabled forever, disarm is already done. */
> > + return 0;
> > + }
> >
> > if (*cnt == 1) {
> > ret = unregister_ftrace_function(ops);
> >
This one, it should fix simply.
Thank you,
> >
>
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
next prev parent reply other threads:[~2025-12-02 2:32 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-11-25 2:05 [PATCH] kprobes: avoid crash when rmmod/insmod modules after ftrace_disabled Ye Bin
2025-11-27 3:52 ` Masami Hiramatsu
2025-11-27 4:18 ` Masami Hiramatsu
2025-11-28 2:27 ` yebin
2025-12-02 2:32 ` Masami Hiramatsu [this message]
2025-12-03 2:37 ` yebin
2025-12-03 4:51 ` Masami Hiramatsu
2025-12-02 16:43 ` Steven Rostedt
2025-12-03 3:21 ` yebin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20251202113248.e9e2e7c665bd3ee07b83e399@kernel.org \
--to=mhiramat@kernel.org \
--cc=davem@davemloft.net \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=naveen@kernel.org \
--cc=rostedt@goodmis.org \
--cc=yebin10@huawei.com \
--cc=yebin@huaweicloud.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).