From: Yafang Shao <laoar.shao@gmail.com>
To: jpoimboe@kernel.org, mbenes@suse.cz, pmladek@suse.com,
joe.lawrence@redhat.com, song@kernel.org
Cc: live-patching@vger.kernel.org, Yafang Shao <laoar.shao@gmail.com>
Subject: [PATCH] livepatch: Avoid blocking tasklist_lock too long
Date: Tue, 17 Nov 2026 21:08:48 +0800 [thread overview]
Message-ID: <20261117130848.65142-1-laoar.shao@gmail.com> (raw)
Problem
=======
As more and more patches accumulate in the cumulative livepatch, we see an
increasing number of latency spikes from latency-sensitive services during
livepatch upgrades. The most recent case is the redis data_server [0].
With some tracing added to the userspace redis code and a hard trace in the
kernel, we finally tracked the root cause down to tasklist_lock contention
during livepatch transition.
When redis flushes data, it clones a new task, which takes the write side of
tasklist_lock. If a livepatch upgrade is in progress at the same time,
tasklist_lock is held for the whole transition. Below is the trace we
captured while reproducing the issue:
=== [2026-11-15 15:18:06] tid=1635309 comm=data_server WRITE wait=79372 us
queued_write_lock_slowpath+276
queued_write_lock_slowpath+276
_raw_write_lock_irq+69
copy_process+3474
kernel_clone+157
__do_sys_clone+102
__x64_sys_clone+37
do_syscall_64+56
entry_SYSCALL_64_after_hwframe+100
=== [2026-11-15 15:18:06] klp scan held tasklist_lock(read) ~ 96864 us (comm=insmod)
klp_enable_patch.cold+247
init_module+477
do_one_initcall+70
do_init_module+82
load_module+2526
__do_sys_finit_module+200
__x64_sys_finit_module+24
do_syscall_64+56
entry_SYSCALL_64_after_hwframe+100
=== [2026-11-15 15:18:08] klp scan held tasklist_lock(read) ~ 3504145 us (comm=kworker/8:2)
klp_transition_work_fn+36
process_one_work+492
worker_thread+83
kthread+235
ret_from_fork+31
crash> p klp_signals_cnt
klp_signals_cnt = $1 = 1
The latency budget of our redis workload is 50 ms, while the clone() call
observed during the livepatch transition took ~79 ms.
Environment used for the reproduction
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
server: 256C AMD EPYC server
tasks: 1000~2000
number of patched functions: around 80
Solution
========
The redis application can work around this issue by removing the clone()
from its data flush path, but I believe we should improve this in the kernel
if we can.
In 2025, we discussed completely replacing tasklist_lock with RCU [1], but
Petr pointed out that it is far more complicated than we expected [2], so we
ended up abandoning that idea.
We still have a more lightweight way to partially replace this heavy
tasklist_lock with RCU, as Josh suggested [3]:
: Maybe klp_try_complete_transition() could iterate the tasks in two
: passes? The first pass would use rcu_read_lock(). Then if all tasks
: appear to be patched, try again with tasklist_lock.
This patch implements that suggestion.
Verification
============
I verified this solution on my test server:
server: 64C AMD virtual server
tasks: ~2000 processes with ~8000 threads, ~30 running
number of patched functions: 200
- Before this patch
the maximum clone() latency during the livepatch transition exceeded
500 ms.
- After this patch
the maximum clone() latency during the livepatch transition is below
20 ms, which is within the range expected by the redis side.
Link: https://github.com/redis/redis [0]
Link: https://lore.kernel.org/all/20250227024733.16989-1-laoar.shao@gmail.com/ [1]
Link: https://lore.kernel.org/all/Z8CRXxfI24E9_IHC@pathway.suse.cz/ [2]
Link: https://lore.kernel.org/all/20250213013603.i6uxtjvc3qxlsqwc@jpoimboe/ [3]
Suggested-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Yafang Shao <laoar.shao@gmail.com>
---
kernel/livepatch/transition.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/kernel/livepatch/transition.c b/kernel/livepatch/transition.c
index 2351a19ac2a9..e058d21d84b7 100644
--- a/kernel/livepatch/transition.c
+++ b/kernel/livepatch/transition.c
@@ -444,7 +444,27 @@ void klp_try_complete_transition(void)
*
* Usually this will transition most (or all) of the tasks on a system
* unless the patch includes changes to a very common function.
+ *
+ * To avoid holding the tasklist_lock for too long, we iterate over the
+ * tasks in two passes. In the first pass, we use the lightweight RCU
+ * read lock to iterate through most tasks. This pass will cover the
+ * majority of tasks, except for some newly forked tasks which may be
+ * added into the tasklist concurrently. In the second pass, we acquire
+ * the tasklist_lock to handle the tasks missed in the first pass.
*/
+
+ /* First pass: Iterate over tasks using the lightweight RCU read lock. */
+ rcu_read_lock();
+ for_each_process_thread(g, task) {
+ /* klp_check_stack() requires preemption to be disabled */
+ if (task == current)
+ continue;
+
+ klp_try_switch_task(task);
+ }
+ rcu_read_unlock();
+
+ /* Second pass: Iterate over missed tasks, holding tasklist_lock. */
read_lock(&tasklist_lock);
for_each_process_thread(g, task)
if (!klp_try_switch_task(task))
--
2.52.0
next reply other threads:[~2026-11-17 13:08 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 13:08 Yafang Shao [this message]
2026-11-17 13:29 ` [PATCH] livepatch: Avoid blocking tasklist_lock too long sashiko-bot
2026-11-17 14:26 ` Yafang Shao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261117130848.65142-1-laoar.shao@gmail.com \
--to=laoar.shao@gmail.com \
--cc=joe.lawrence@redhat.com \
--cc=jpoimboe@kernel.org \
--cc=live-patching@vger.kernel.org \
--cc=mbenes@suse.cz \
--cc=pmladek@suse.com \
--cc=song@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox