* [PATCH v5] tracing: Fix race between update_event_fields and, event_define_fields
@ 2026-08-10 6:32 Michael Wu
2026-08-10 14:45 ` Steven Rostedt
0 siblings, 1 reply; 3+ messages in thread
From: Michael Wu @ 2026-08-10 6:32 UTC (permalink / raw)
To: Steven Rostedt, Masami Hiramatsu
Cc: Mathieu Desnoyers, linux-kernel, linux-trace-kernel
The following sequence may leads race between event_define_fields()
and update_event_fields():
CPU0 (module A, pri=1 notifier) CPU1 (module B, pri=0 notifier)
=============================== ===============================
event_define_fields(call_A) trace_event_update_all()
for each f: list_for_each_entry(...,
list_add(&f->link, &ftrace_events)
&class->fields) -> finds call_A
f->link.next = next; (2)
update_event_fields(call_A)
WRITE_ONCE(class->fields->next,
&f->link); (4)
list_for_each_entry(field,
&class->fields, link)
-> field = class->fields->next
= &f->link
= f (offset 0)
-> arm64 weak ordering:
(4) visible before (2)
field->link.next == 0
-> next iteration:
field = (void *)0 = NULL
-> crash at NULL->type (0x18)
This produces the following panic:
Unable to handle kernel access ... at virtual address 0000000000000018
pc : update_event_fields+0xf8/0x368
Call trace:
update_event_fields+0xf8/0x368
trace_event_update_all+0x7c/0x2b4
trace_module_notify+0x4c/0x1dc
notifier_call_chain+0x84/0x168
blocking_notifier_call_chain_robust+0x64/0xd4
load_module+0x10c8/0x123c
__arm64_sys_finit_module+0x230/0x31c
Fix by taking event_mutex in trace_event_update_all() before
trace_event_sem.
Fixes: b3bc8547d3be ("tracing: Have TRACE_DEFINE_ENUM affect trace event types as well")
Cc: stable@vger.kernel.org
Signed-off-by: Michael Wu <michael@allwinnertech.com>
---
kernel/trace/trace_events.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index 956692856fa8..9632788da5af 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -3566,6 +3566,7 @@ void trace_event_update_all(struct trace_eval_map **map, int len)
int last_i;
int i;
+ mutex_lock(&event_mutex);
down_write(&trace_event_sem);
list_for_each_entry_safe(call, p, &ftrace_events, list) {
/* events are usually grouped together with systems */
@@ -3604,6 +3605,7 @@ void trace_event_update_all(struct trace_eval_map **map, int len)
cond_resched();
}
up_write(&trace_event_sem);
+ mutex_unlock(&event_mutex);
}
static bool event_in_systems(struct trace_event_call *call,
--
2.29.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v5] tracing: Fix race between update_event_fields and, event_define_fields
2026-08-10 6:32 [PATCH v5] tracing: Fix race between update_event_fields and, event_define_fields Michael Wu
@ 2026-08-10 14:45 ` Steven Rostedt
2026-08-11 6:00 ` Michael Wu
0 siblings, 1 reply; 3+ messages in thread
From: Steven Rostedt @ 2026-08-10 14:45 UTC (permalink / raw)
To: Michael Wu
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel
On Mon, 10 Aug 2026 14:32:30 +0800
Michael Wu <michael@allwinnertech.com> wrote:
> The following sequence may leads race between event_define_fields()
> and update_event_fields():
> CPU0 (module A, pri=1 notifier) CPU1 (module B, pri=0 notifier)
What does the above mean? Are you loading two modules at the same time?
What does "pri=X notifier" mean? What function calls are these coming from?
-- Steve
> =============================== ===============================
> event_define_fields(call_A) trace_event_update_all()
> for each f: list_for_each_entry(...,
> list_add(&f->link, &ftrace_events)
> &class->fields) -> finds call_A
> f->link.next = next; (2)
> update_event_fields(call_A)
> WRITE_ONCE(class->fields->next,
> &f->link); (4)
> list_for_each_entry(field,
> &class->fields, link)
> -> field = class->fields->next
> = &f->link
> = f (offset 0)
> -> arm64 weak ordering:
> (4) visible before (2)
> field->link.next == 0
> -> next iteration:
> field = (void *)0 = NULL
> -> crash at NULL->type (0x18)
>
> This produces the following panic:
> Unable to handle kernel access ... at virtual address 0000000000000018
> pc : update_event_fields+0xf8/0x368
> Call trace:
> update_event_fields+0xf8/0x368
> trace_event_update_all+0x7c/0x2b4
> trace_module_notify+0x4c/0x1dc
> notifier_call_chain+0x84/0x168
> blocking_notifier_call_chain_robust+0x64/0xd4
> load_module+0x10c8/0x123c
> __arm64_sys_finit_module+0x230/0x31c
>
> Fix by taking event_mutex in trace_event_update_all() before
> trace_event_sem.
>
> Fixes: b3bc8547d3be ("tracing: Have TRACE_DEFINE_ENUM affect trace event types as well")
> Cc: stable@vger.kernel.org
> Signed-off-by: Michael Wu <michael@allwinnertech.com>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v5] tracing: Fix race between update_event_fields and, event_define_fields
2026-08-10 14:45 ` Steven Rostedt
@ 2026-08-11 6:00 ` Michael Wu
0 siblings, 0 replies; 3+ messages in thread
From: Michael Wu @ 2026-08-11 6:00 UTC (permalink / raw)
To: Steven Rostedt
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel
> What does the above mean? Are you loading two modules at the same time?
Two modules (A and B) are loaded simultaneously on different CPUs. On the arm64,
when CPU0's trace_module_notify [pri=1] and CPU1's trace_module_notify [pri=0]
simultaneously perform operations on call_A, because they are in different cache lines,
CPU1 may observe WRITE_ONCE(head->next, &f->link) in step (4) before f->link.next=next in step (2).
At this time, CPU1 reads an uninitialized f->link.next and performs an operation that causes to crash.
> What does "pri=X notifier" mean? What function calls are these coming from?
`pri=X notifier` represents `trace_events.c:trace_module_notify [pri=1]` and `trace.c:trace_module_notify [pri=0]`, respectively.
CPU0 (loads module A) CPU1 (loads module B)
=============================== ===============================
load_module(A) load_module(B)
blocking_notifier_call_chain_robust blocking_notifier_call_chain_robust
notifier_call_chain notifier_call_chain
nb = trace_events.c: nb = trace.c:
trace_module_notify [pri=1] trace_module_notify [pri=0]
mutex_lock(&event_mutex) trace_event_update_all()
trace_module_add_events(A) down_write(&trace_event_sem)
__register_event(call_A)
__add_event_to_tracers(call_A)
event_define_fields(call_A)
for each f:
f = kmem_cache_alloc()
list_add(&f->link,
&class->fields)
f->link.next=next; (2)
WRITE_ONCE(head->next,
&f->link); (4) update_event_fields(call_A)
mutex_unlock(&event_mutex) list_for_each_entry(field,
&class->fields, link)
field = class->fields->next
= &f->link
= f (offset 0)
up_write(&trace_event_sem)
On 8/10/2026 10:45 PM, Steven Rostedt wrote:
> On Mon, 10 Aug 2026 14:32:30 +0800
> Michael Wu <michael@allwinnertech.com> wrote:
>
>> The following sequence may leads race between event_define_fields()
>> and update_event_fields():
>
>> CPU0 (module A, pri=1 notifier) CPU1 (module B, pri=0 notifier)
>
> What does the above mean? Are you loading two modules at the same time?
> What does "pri=X notifier" mean? What function calls are these coming from?
>
> -- Steve
>
>> =============================== ===============================
>> event_define_fields(call_A) trace_event_update_all()
>> for each f: list_for_each_entry(...,
>> list_add(&f->link, &ftrace_events)
>> &class->fields) -> finds call_A
>> f->link.next = next; (2)
>> update_event_fields(call_A)
>> WRITE_ONCE(class->fields->next,
>> &f->link); (4)
>> list_for_each_entry(field,
>> &class->fields, link)
>> -> field = class->fields->next
>> = &f->link
>> = f (offset 0)
>> -> arm64 weak ordering:
>> (4) visible before (2)
>> field->link.next == 0
>> -> next iteration:
>> field = (void *)0 = NULL
>> -> crash at NULL->type (0x18)
>>
>> This produces the following panic:
>> Unable to handle kernel access ... at virtual address 0000000000000018
>> pc : update_event_fields+0xf8/0x368
>> Call trace:
>> update_event_fields+0xf8/0x368
>> trace_event_update_all+0x7c/0x2b4
>> trace_module_notify+0x4c/0x1dc
>> notifier_call_chain+0x84/0x168
>> blocking_notifier_call_chain_robust+0x64/0xd4
>> load_module+0x10c8/0x123c
>> __arm64_sys_finit_module+0x230/0x31c
>>
>> Fix by taking event_mutex in trace_event_update_all() before
>> trace_event_sem.
>>
>> Fixes: b3bc8547d3be ("tracing: Have TRACE_DEFINE_ENUM affect trace event types as well")
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Michael Wu <michael@allwinnertech.com>
--
Regards,
Michael Wu
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-08-11 6:00 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-10 6:32 [PATCH v5] tracing: Fix race between update_event_fields and, event_define_fields Michael Wu
2026-08-10 14:45 ` Steven Rostedt
2026-08-11 6:00 ` Michael Wu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox