From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org,
syzbot+03beeb595f074db9cfd1@syzkaller.appspotmail.com,
Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>,
Jens Axboe <axboe@kernel.dk>
Subject: [PATCH 5.10 38/63] io_uring: check kthread stopped flag when sq thread is unparked
Date: Mon, 4 Jan 2021 16:57:31 +0100 [thread overview]
Message-ID: <20210104155710.668728286@linuxfoundation.org> (raw)
In-Reply-To: <20210104155708.800470590@linuxfoundation.org>
From: Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
commit 65b2b213484acd89a3c20dbb524e52a2f3793b78 upstream.
syzbot reports following issue:
INFO: task syz-executor.2:12399 can't die for more than 143 seconds.
task:syz-executor.2 state:D stack:28744 pid:12399 ppid: 8504 flags:0x00004004
Call Trace:
context_switch kernel/sched/core.c:3773 [inline]
__schedule+0x893/0x2170 kernel/sched/core.c:4522
schedule+0xcf/0x270 kernel/sched/core.c:4600
schedule_timeout+0x1d8/0x250 kernel/time/timer.c:1847
do_wait_for_common kernel/sched/completion.c:85 [inline]
__wait_for_common kernel/sched/completion.c:106 [inline]
wait_for_common kernel/sched/completion.c:117 [inline]
wait_for_completion+0x163/0x260 kernel/sched/completion.c:138
kthread_stop+0x17a/0x720 kernel/kthread.c:596
io_put_sq_data fs/io_uring.c:7193 [inline]
io_sq_thread_stop+0x452/0x570 fs/io_uring.c:7290
io_finish_async fs/io_uring.c:7297 [inline]
io_sq_offload_create fs/io_uring.c:8015 [inline]
io_uring_create fs/io_uring.c:9433 [inline]
io_uring_setup+0x19b7/0x3730 fs/io_uring.c:9507
do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x45deb9
Code: Unable to access opcode bytes at RIP 0x45de8f.
RSP: 002b:00007f174e51ac78 EFLAGS: 00000246 ORIG_RAX: 00000000000001a9
RAX: ffffffffffffffda RBX: 0000000000008640 RCX: 000000000045deb9
RDX: 0000000000000000 RSI: 0000000020000140 RDI: 00000000000050e5
RBP: 000000000118bf58 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000118bf2c
R13: 00007ffed9ca723f R14: 00007f174e51b9c0 R15: 000000000118bf2c
INFO: task syz-executor.2:12399 blocked for more than 143 seconds.
Not tainted 5.10.0-rc3-next-20201110-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Currently we don't have a reproducer yet, but seems that there is a
race in current codes:
=> io_put_sq_data
ctx_list is empty now. |
==> kthread_park(sqd->thread); |
| T1: sq thread is parked now.
==> kthread_stop(sqd->thread); |
KTHREAD_SHOULD_STOP is set now.|
===> kthread_unpark(k); |
| T2: sq thread is now unparkd, run again.
|
| T3: sq thread is now preempted out.
|
===> wake_up_process(k); |
|
| T4: Since sqd ctx_list is empty, needs_sched will be true,
| then sq thread sets task state to TASK_INTERRUPTIBLE,
| and schedule, now sq thread will never be waken up.
===> wait_for_completion |
I have artificially used mdelay() to simulate above race, will get same
stack like this syzbot report, but to be honest, I'm not sure this code
race triggers syzbot report.
To fix this possible code race, when sq thread is unparked, need to check
whether sq thread has been stopped.
Reported-by: syzbot+03beeb595f074db9cfd1@syzkaller.appspotmail.com
Signed-off-by: Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
fs/io_uring.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -6802,8 +6802,16 @@ static int io_sq_thread(void *data)
* kthread parking. This synchronizes the thread vs users,
* the users are synchronized on the sqd->ctx_lock.
*/
- if (kthread_should_park())
+ if (kthread_should_park()) {
kthread_parkme();
+ /*
+ * When sq thread is unparked, in case the previous park operation
+ * comes from io_put_sq_data(), which means that sq thread is going
+ * to be stopped, so here needs to have a check.
+ */
+ if (kthread_should_stop())
+ break;
+ }
if (unlikely(!list_empty(&sqd->ctx_new_list)))
io_sqd_init_new(sqd);
next prev parent reply other threads:[~2021-01-04 16:04 UTC|newest]
Thread overview: 75+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-04 15:56 [PATCH 5.10 00/63] 5.10.5-rc1 review Greg Kroah-Hartman
2021-01-04 15:56 ` [PATCH 5.10 01/63] net/sched: sch_taprio: reset child qdiscs before freeing them Greg Kroah-Hartman
2021-01-04 22:58 ` Sasha Levin
2021-01-04 23:06 ` Jakub Kicinski
2021-01-04 15:56 ` [PATCH 5.10 02/63] mptcp: fix security context on server socket Greg Kroah-Hartman
2021-01-04 15:56 ` [PATCH 5.10 03/63] ethtool: fix error paths in ethnl_set_channels() Greg Kroah-Hartman
2021-01-04 15:56 ` [PATCH 5.10 04/63] ethtool: fix string set id check Greg Kroah-Hartman
2021-01-04 15:56 ` [PATCH 5.10 05/63] md/raid10: initialize r10_bio->read_slot before use Greg Kroah-Hartman
2021-01-04 15:56 ` [PATCH 5.10 06/63] drm/amd/display: Add get_dig_frontend implementation for DCEx Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 07/63] io_uring: close a small race gap for files cancel Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 08/63] jffs2: Allow setting rp_size to zero during remounting Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 09/63] jffs2: Fix NULL pointer dereference in rp_size fs option parsing Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 10/63] spi: dw-bt1: Fix undefined devm_mux_control_get symbol Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 11/63] opp: fix memory leak in _allocate_opp_table Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 12/63] opp: Call the missing clk_put() on error Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 13/63] scsi: block: Fix a race in the runtime power management code Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 14/63] mm/hugetlb: fix deadlock in hugetlb_cow error path Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 15/63] mm: memmap defer init doesnt work as expected Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 16/63] lib/zlib: fix inflating zlib streams on s390 Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 17/63] io_uring: dont assume mm is constant across submits Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 18/63] io_uring: use bottom half safe lock for fixed file data Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 19/63] io_uring: add a helper for setting a ref node Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 20/63] io_uring: fix io_sqe_files_unregister() hangs Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 21/63] kernel/io_uring: cancel io_uring before task works Greg Kroah-Hartman
2021-01-04 16:06 ` Pavel Begunkov
2021-01-04 17:43 ` Sasha Levin
2021-01-04 15:57 ` [PATCH 5.10 22/63] uapi: move constants from <linux/kernel.h> to <linux/const.h> Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 23/63] tools headers UAPI: Sync linux/const.h with the kernel headers Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 24/63] cgroup: Fix memory leak when parsing multiple source parameters Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 25/63] zlib: move EXPORT_SYMBOL() and MODULE_LICENSE() out of dfltcc_syms.c Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 26/63] scsi: cxgb4i: Fix TLS dependency Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 27/63] Bluetooth: hci_h5: close serdev device and free hu in h5_close Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 28/63] fbcon: Disable accelerated scrolling Greg Kroah-Hartman
2021-01-07 8:13 ` Geert Uytterhoeven
2021-01-04 15:57 ` [PATCH 5.10 29/63] reiserfs: add check for an invalid ih_entry_count Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 30/63] misc: vmw_vmci: fix kernel info-leak by initializing dbells in vmci_ctx_get_chkpt_doorbells() Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 31/63] media: gp8psk: initialize stats at power control logic Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 32/63] f2fs: fix shift-out-of-bounds in sanity_check_raw_super() Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 33/63] ALSA: seq: Use bool for snd_seq_queue internal flags Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 34/63] ALSA: rawmidi: Access runtime->avail always in spinlock Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 35/63] bfs: dont use WARNING: string when its just info Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 36/63] ext4: check for invalid block size early when mounting a file system Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 37/63] fcntl: Fix potential deadlock in send_sig{io, urg}() Greg Kroah-Hartman
2021-01-04 15:57 ` Greg Kroah-Hartman [this message]
2021-01-04 15:57 ` [PATCH 5.10 39/63] rtc: sun6i: Fix memleak in sun6i_rtc_clk_init Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 40/63] module: set MODULE_STATE_GOING state when a module fails to load Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 41/63] quota: Dont overflow quota file offsets Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 42/63] rtc: pl031: fix resource leak in pl031_probe Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 43/63] powerpc: sysdev: add missing iounmap() on error in mpic_msgr_probe() Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 44/63] i3c master: fix missing destroy_workqueue() on error in i3c_master_register Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 45/63] NFSv4: Fix a pNFS layout related use-after-free race when freeing the inode Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 46/63] f2fs: avoid race condition for shrinker count Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 47/63] f2fs: fix race of pending_pages in decompression Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 48/63] module: delay kobject uevent until after module init call Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 49/63] powerpc/64: irq replay remove decrementer overflow check Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 50/63] fs/namespace.c: WARN if mnt_count has become negative Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 51/63] watchdog: rti-wdt: fix reference leak in rti_wdt_probe Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 52/63] um: random: Register random as hwrng-core device Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 53/63] um: ubd: Submit all data segments atomically Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 54/63] NFSv4.2: Dont error when exiting early on a READ_PLUS buffer overflow Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 55/63] ceph: fix inode refcount leak when ceph_fill_inode on non-I_NEW inode fails Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 56/63] drm/amd/display: updated wm table for Renoir Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 57/63] tick/sched: Remove bogus boot "safety" check Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 58/63] s390: always clear kernel stack backchain before calling functions Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 59/63] io_uring: remove racy overflow list fast checks Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 60/63] ALSA: pcm: Clear the full allocated memory at hw_params Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 61/63] dm verity: skip verity work if I/O error when system is shutting down Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 62/63] ext4: avoid s_mb_prefetch to be zero in individual scenarios Greg Kroah-Hartman
2021-01-04 15:57 ` [PATCH 5.10 63/63] device-dax: Fix range release Greg Kroah-Hartman
2021-01-05 6:06 ` [PATCH 5.10 00/63] 5.10.5-rc1 review Daniel Díaz
2021-01-05 12:55 ` Jeffrin Jose T
2021-01-05 13:05 ` Greg Kroah-Hartman
2021-01-06 18:43 ` Jeffrin Jose T
2021-01-05 16:38 ` Shuah Khan
2021-01-05 18:17 ` Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210104155710.668728286@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=axboe@kernel.dk \
--cc=linux-kernel@vger.kernel.org \
--cc=stable@vger.kernel.org \
--cc=syzbot+03beeb595f074db9cfd1@syzkaller.appspotmail.com \
--cc=xiaoguang.wang@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox