From: Tejun Heo <tj@kernel.org>
To: void@manifault.com
Cc: kernel-team@meta.com, linux-kernel@vger.kernel.org,
sched-ext@meta.com, aboorvad@linux.ibm.com
Subject: [PATCHSET sched_ext/for-6.12-fixes] sched_ext: Fix locking enable/disable path bugs includling locking order one
Date: Mon, 23 Sep 2024 08:59:27 -1000 [thread overview]
Message-ID: <20240923190020.1446325-1-tj@kernel.org> (raw)
Aboorva Devarajan reported an issue where sched_ext init code can
occasionally deadlock when scheduler loading races CPU hotplug. The deadlock
scenario is as follows:
scx_ops_enable() hotplug
percpu_down_write(&cpu_hotplug_lock)
percpu_down_write(&scx_fork_rwsem)
block on cpu_hotplug_lock
kthread_create() waits for kthreadd
kthreadd blocks on scx_fork_rwsem
Note that this doesn't trigger lockdep because the hotplug side dependency
bounces through kthreadd.
This is primarily caused by SCX enable/disable paths grabbing big locks
together. This patchset updates the enable/disable paths to decouple the
locks. In the process, it also fixes several subtle bugs in the enable path.
This patchset contains the following patches:
0001-sched_ext-Relocate-check_hotplug_seq-call-in-scx_ops.patch
0002-sched_ext-Remove-SCX_OPS_PREPPING.patch
0003-sched_ext-Initialize-in-bypass-mode.patch
0004-sched_ext-Fix-SCX_TASK_INIT-SCX_TASK_READY-transitio.patch
0005-sched_ext-Enable-scx_ops_init_task-separately.patch
0006-sched_ext-Add-scx_cgroup_enabled-to-gate-cgroup-oper.patch
0007-sched_ext-Decouple-locks-in-scx_ops_disable_workfn.patch
0008-sched_ext-Decouple-locks-in-scx_ops_enable.patch
0001-0002 are prep patches.
0003 removes a race window in the enable path that can cause stalls and
prepares for further locking updates.
0004-0005 remove race windows in the enable path that can cause invalid task
state transitions.
0006 fixes a bug in cgroup enable path which can skip invocation of
ops.cgroup_exit() and prepares for further locking updates.
0007-0008 decouple the big locks and fix the deadlock.
This patchset can also be found in the following git branch:
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git scx-enable-locking-fix
diffstat follows. Thanks.
kernel/sched/ext.c | 199 ++++++++++++++++++++++++++++++++------------------------------------
1 file changed, 94 insertions(+), 105 deletions(-)
--
tejun
next reply other threads:[~2024-09-23 19:00 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-09-23 18:59 Tejun Heo [this message]
2024-09-23 18:59 ` [PATCH 1/8] sched_ext: Relocate check_hotplug_seq() call in scx_ops_enable() Tejun Heo
2024-09-23 18:59 ` [PATCH 2/8] sched_ext: Remove SCX_OPS_PREPPING Tejun Heo
2024-09-23 18:59 ` [PATCH 3/8] sched_ext: Initialize in bypass mode Tejun Heo
2024-09-23 18:59 ` [PATCH 4/8] sched_ext: Fix SCX_TASK_INIT -> SCX_TASK_READY transitions in scx_ops_enable() Tejun Heo
2024-09-23 18:59 ` [PATCH 5/8] sched_ext: Enable scx_ops_init_task() separately Tejun Heo
2024-09-23 18:59 ` [PATCH 6/8] sched_ext: Add scx_cgroup_enabled to gate cgroup operations and fix scx_tg_online() Tejun Heo
2024-09-23 18:59 ` [PATCH 7/8] sched_ext: Decouple locks in scx_ops_disable_workfn() Tejun Heo
2024-09-23 18:59 ` [PATCH 8/8] sched_ext: Decouple locks in scx_ops_enable() Tejun Heo
2024-09-27 20:03 ` [PATCHSET sched_ext/for-6.12-fixes] sched_ext: Fix locking enable/disable path bugs includling locking order one Tejun Heo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240923190020.1446325-1-tj@kernel.org \
--to=tj@kernel.org \
--cc=aboorvad@linux.ibm.com \
--cc=kernel-team@meta.com \
--cc=linux-kernel@vger.kernel.org \
--cc=sched-ext@meta.com \
--cc=void@manifault.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.