From: Steven Rostedt <rostedt@goodmis.org>
To: linux-kernel@vger.kernel.org
Cc: Daniel Bristot de Oliveira <bristot@kernel.org>,
Nico Pache <npache@redhat.com>
Subject: [for-next][PATCH 14/20] tracing/osnoise: Fix possible recursive locking in stop_per_cpu_kthreads
Date: Tue, 27 Sep 2022 12:02:30 -0400 [thread overview]
Message-ID: <20220927160248.807986897@goodmis.org> (raw)
In-Reply-To: 20220927160216.349640304@goodmis.org
From: Nico Pache <npache@redhat.com>
There is a recursive lock on the cpu_hotplug_lock.
In kernel/trace/trace_osnoise.c:<start/stop>_per_cpu_kthreads:
- start_per_cpu_kthreads calls cpus_read_lock() and if
start_kthreads returns a error it will call stop_per_cpu_kthreads.
- stop_per_cpu_kthreads then calls cpus_read_lock() again causing
deadlock.
Fix this by calling cpus_read_unlock() before calling
stop_per_cpu_kthreads. This behavior can also be seen in commit
f46b16520a08 ("trace/hwlat: Implement the per-cpu mode").
This error was noticed during the LTP ftrace-stress-test:
WARNING: possible recursive locking detected
--------------------------------------------
sh/275006 is trying to acquire lock:
ffffffffb02f5400 (cpu_hotplug_lock){++++}-{0:0}, at: stop_per_cpu_kthreads
but task is already holding lock:
ffffffffb02f5400 (cpu_hotplug_lock){++++}-{0:0}, at: start_per_cpu_kthreads
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
----
lock(cpu_hotplug_lock);
lock(cpu_hotplug_lock);
*** DEADLOCK ***
May be due to missing lock nesting notation
3 locks held by sh/275006:
#0: ffff8881023f0470 (sb_writers#24){.+.+}-{0:0}, at: ksys_write
#1: ffffffffb084f430 (trace_types_lock){+.+.}-{3:3}, at: rb_simple_write
#2: ffffffffb02f5400 (cpu_hotplug_lock){++++}-{0:0}, at: start_per_cpu_kthreads
Link: https://lkml.kernel.org/r/20220919144932.3064014-1-npache@redhat.com
Fixes: c8895e271f79 ("trace/osnoise: Support hotplug operations")
Signed-off-by: Nico Pache <npache@redhat.com>
Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
kernel/trace/trace_osnoise.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c
index 313439920a8c..78d536d3ff3d 100644
--- a/kernel/trace/trace_osnoise.c
+++ b/kernel/trace/trace_osnoise.c
@@ -1786,8 +1786,9 @@ static int start_per_cpu_kthreads(void)
for_each_cpu(cpu, current_mask) {
retval = start_kthread(cpu);
if (retval) {
+ cpus_read_unlock();
stop_per_cpu_kthreads();
- break;
+ return retval;
}
}
--
2.35.1
next prev parent reply other threads:[~2022-09-27 16:02 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-27 16:02 [for-next][PATCH 00/20] tracing: Update for 6.1 Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 01/20] tracing/eprobe: Add eprobe filter support Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 02/20] selftests/ftrace: Add eprobe syntax error testcase Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 03/20] rv/monitors: add static qualifier for local symbols Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 04/20] rv/dot2K: add static qualifier for local variable Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 05/20] tracing: Add numeric delta time to the trace event benchmark Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 06/20] tracing/hist: Call hist functions directly via a switch statement Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 07/20] tracing: Move struct filter_pred into trace_events_filter.c Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 08/20] tracing/filter: Call filter predicate functions directly via a switch statement Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 09/20] tracepoint: Optimize the critical region of mutex_lock in tracepoint_module_coming() Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 10/20] x86/ftrace: Remove unused modifying_ftrace_code declaration Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 11/20] x86/kprobes: Remove unused arch_kprobe_override_function() declaration Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 12/20] tracing: kprobe: Fix kprobe event gen test module on exit Steven Rostedt
2022-09-27 16:02 ` Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 13/20] tracing: kprobe: Make gen test module work in arm and riscv Steven Rostedt
2022-09-27 16:02 ` Steven Rostedt
2022-09-27 16:02 ` Steven Rostedt [this message]
2022-09-27 16:02 ` [for-next][PATCH 15/20] rv/monitor: Add __init/__exit annotations to module init/exit funcs Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 16/20] tracing: Disable interrupt or preemption before acquiring arch_spinlock_t Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 17/20] ftrace: Remove obsoleted code from ftrace and task_struct Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 18/20] x86: kprobes: Remove unused macro stack_addr Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 19/20] tracing/eprobe: Fix alloc event dir failed when event name no set Steven Rostedt
2022-09-27 16:02 ` [for-next][PATCH 20/20] ftrace: Properly unset FTRACE_HASH_FL_MOD Steven Rostedt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220927160248.807986897@goodmis.org \
--to=rostedt@goodmis.org \
--cc=bristot@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=npache@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.