From: Tejun Heo <tj@kernel.org>
To: void@manifault.com
Cc: kernel-team@meta.com, linux-kernel@vger.kernel.org,
sched-ext@meta.com, aboorvad@linux.ibm.com
Subject: [PATCHSET sched_ext/for-6.12-fixes] sched_ext: Fix locking enable/disable path bugs includling locking order one
Date: Mon, 23 Sep 2024 08:59:27 -1000 [thread overview]
Message-ID: <20240923190020.1446325-1-tj@kernel.org> (raw)
Aboorva Devarajan reported an issue where sched_ext init code can
occasionally deadlock when scheduler loading races CPU hotplug. The deadlock
scenario is as follows:
scx_ops_enable() hotplug
percpu_down_write(&cpu_hotplug_lock)
percpu_down_write(&scx_fork_rwsem)
block on cpu_hotplug_lock
kthread_create() waits for kthreadd
kthreadd blocks on scx_fork_rwsem
Note that this doesn't trigger lockdep because the hotplug side dependency
bounces through kthreadd.
This is primarily caused by SCX enable/disable paths grabbing big locks
together. This patchset updates the enable/disable paths to decouple the
locks. In the process, it also fixes several subtle bugs in the enable path.
This patchset contains the following patches:
0001-sched_ext-Relocate-check_hotplug_seq-call-in-scx_ops.patch
0002-sched_ext-Remove-SCX_OPS_PREPPING.patch
0003-sched_ext-Initialize-in-bypass-mode.patch
0004-sched_ext-Fix-SCX_TASK_INIT-SCX_TASK_READY-transitio.patch
0005-sched_ext-Enable-scx_ops_init_task-separately.patch
0006-sched_ext-Add-scx_cgroup_enabled-to-gate-cgroup-oper.patch
0007-sched_ext-Decouple-locks-in-scx_ops_disable_workfn.patch
0008-sched_ext-Decouple-locks-in-scx_ops_enable.patch
0001-0002 are prep patches.
0003 removes a race window in the enable path that can cause stalls and
prepares for further locking updates.
0004-0005 remove race windows in the enable path that can cause invalid task
state transitions.
0006 fixes a bug in cgroup enable path which can skip invocation of
ops.cgroup_exit() and prepares for further locking updates.
0007-0008 decouple the big locks and fix the deadlock.
This patchset can also be found in the following git branch:
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git scx-enable-locking-fix
diffstat follows. Thanks.
kernel/sched/ext.c | 199 ++++++++++++++++++++++++++++++++------------------------------------
1 file changed, 94 insertions(+), 105 deletions(-)
--
tejun
next reply other threads:[~2024-09-23 19:00 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-09-23 18:59 Tejun Heo [this message]
2024-09-23 18:59 ` [PATCH 1/8] sched_ext: Relocate check_hotplug_seq() call in scx_ops_enable() Tejun Heo
2024-09-23 18:59 ` [PATCH 2/8] sched_ext: Remove SCX_OPS_PREPPING Tejun Heo
2024-09-23 18:59 ` [PATCH 3/8] sched_ext: Initialize in bypass mode Tejun Heo
2024-09-23 18:59 ` [PATCH 4/8] sched_ext: Fix SCX_TASK_INIT -> SCX_TASK_READY transitions in scx_ops_enable() Tejun Heo
2024-09-23 18:59 ` [PATCH 5/8] sched_ext: Enable scx_ops_init_task() separately Tejun Heo
2024-09-23 18:59 ` [PATCH 6/8] sched_ext: Add scx_cgroup_enabled to gate cgroup operations and fix scx_tg_online() Tejun Heo
2024-09-23 18:59 ` [PATCH 7/8] sched_ext: Decouple locks in scx_ops_disable_workfn() Tejun Heo
2024-09-23 18:59 ` [PATCH 8/8] sched_ext: Decouple locks in scx_ops_enable() Tejun Heo
2024-09-27 20:03 ` [PATCHSET sched_ext/for-6.12-fixes] sched_ext: Fix locking enable/disable path bugs includling locking order one Tejun Heo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240923190020.1446325-1-tj@kernel.org \
--to=tj@kernel.org \
--cc=aboorvad@linux.ibm.com \
--cc=kernel-team@meta.com \
--cc=linux-kernel@vger.kernel.org \
--cc=sched-ext@meta.com \
--cc=void@manifault.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox