From: Jens Axboe <axboe@kernel.dk>
To: stable@vger.kernel.org
Subject: [PATCH 7.3.y 5/9] io_uring/io-wq: show stuck workers rather than WARN on a slow exit
Date: Sat, 21 Nov 2026 21:31:50 +0000 [thread overview]
Message-ID: <20261121213154.1790334-6-axboe@kernel.dk> (raw)
In-Reply-To: <20261121213154.1790334-1-axboe@kernel.dk>
Commit d4912d6440bf6d255e1fa1c360c8ddb39d0e01bd upstream.
io_wq_exit_workers() triggers a WARN_ON_ONCE() if it has waited more than
IO_URING_EXIT_WAIT_MAX for the workers to exit. That can be triggered at
will, for example with an open on a FUSE mount where the server never
replies to the lookup. The worker can't be canceled, and the exit waits
until the server goes away.
The stack trace from the WARN isn't useful either, as it's that of the
exiting task and that is always the task exit, exec, or SQPOLL thread
exit path. Print a message and show the workers that are still around
instead, that's where the interesting stack traces are.
Reported-by: syzbot+b0d54b9e81de55179e47@syzkaller.appspotmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
io_uring/io-wq.c | 24 +++++++++++++++++++++---
1 file changed, 21 insertions(+), 3 deletions(-)
diff --git a/io_uring/io-wq.c b/io_uring/io-wq.c
index 6b9395060262..358f32888a85 100644
--- a/io_uring/io-wq.c
+++ b/io_uring/io-wq.c
@@ -9,6 +9,7 @@
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/sched/signal.h>
+#include <linux/sched/debug.h>
#include <linux/percpu.h>
#include <linux/slab.h>
#include <linux/rculist_nulls.h>
@@ -1444,6 +1445,22 @@ static void io_wq_cancel_tw_create(struct io_wq *wq)
}
}
+static bool io_wq_worker_show(struct io_worker *worker, void *data)
+{
+ sched_show_task(worker->task);
+ return false;
+}
+
+/* The exit path is always the same, show what the workers are stuck on */
+static void io_wq_show_stuck_workers(struct io_wq *wq)
+{
+ pr_warn("io-wq: %s/%d stuck waiting for workers to exit\n",
+ current->comm, task_pid_nr(current));
+ rcu_read_lock();
+ io_wq_for_each_worker(wq, io_wq_worker_show, NULL);
+ rcu_read_unlock();
+}
+
static void io_wq_exit_workers(struct io_wq *wq)
{
unsigned long timeout, warn_timeout;
@@ -1465,8 +1482,8 @@ static void io_wq_exit_workers(struct io_wq *wq)
*
* where completely overloading the system with tons of long running
* io-wq items can easily trigger the hung task timeout. Only sleep
- * uninterruptibly for half that time, and warn if we exceeded end
- * up waiting more than IO_URING_EXIT_WAIT_MAX.
+ * uninterruptibly for half that time, and show the stuck workers if
+ * we end up waiting more than IO_URING_EXIT_WAIT_MAX.
*/
timeout = sysctl_hung_task_timeout_secs * HZ / 2;
if (!timeout)
@@ -1475,7 +1492,8 @@ static void io_wq_exit_workers(struct io_wq *wq)
do {
if (wait_for_completion_timeout(&wq->worker_done, timeout))
break;
- WARN_ON_ONCE(time_after(jiffies, warn_timeout));
+ if (time_after(jiffies, warn_timeout))
+ DO_ONCE_LITE(io_wq_show_stuck_workers, wq);
} while (1);
spin_lock_irq(&wq->hash->wait.lock);
--
2.55.0
next prev parent reply other threads:[~2026-11-21 21:32 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-21 21:31 [PATCH 7.3.y 0/9] io_uring fixes for 7.3-stable Jens Axboe
2026-11-21 21:31 ` [PATCH 7.3.y 1/9] io_uring/napi: don't end the single NAPI busy poll early on need_resched() Jens Axboe
2026-11-21 21:31 ` [PATCH 7.3.y 2/9] io_uring: fail soft links on negative IOU_COMPLETE results Jens Axboe
2026-11-21 21:31 ` [PATCH 7.3.y 3/9] io_uring/futex: fail soft links when cancelling a wait Jens Axboe
2026-11-21 21:31 ` [PATCH 7.3.y 4/9] io_uring/rw: fail soft links on IOPOLL short or error IO Jens Axboe
2026-11-21 21:31 ` Jens Axboe [this message]
2026-11-21 21:31 ` [PATCH 7.3.y 6/9] io_uring/timeout: don't rearm a multishot timeout on cancel Jens Axboe
2026-11-21 21:31 ` [PATCH 7.3.y 7/9] io_uring/rsrc: keep the old file when a file update fails Jens Axboe
2026-11-21 21:31 ` [PATCH 7.3.y 8/9] io_uring/kbuf: initialize arg->out_len from arg->max_len in io_buffers_select() Jens Axboe
2026-11-21 21:31 ` [PATCH 7.3.y 9/9] io_uring/io-wq: apply the current cpuset to new workers Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261121213154.1790334-6-axboe@kernel.dk \
--to=axboe@kernel.dk \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox