* [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable
@ 2026-11-21 21:31 Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 1/9] io_uring: enable toggle of iowait usage when waiting on CQEs Jens Axboe
` (8 more replies)
0 siblings, 9 replies; 10+ messages in thread
From: Jens Axboe @ 2026-11-21 21:31 UTC (permalink / raw)
To: stable
Hi Greg,
Here are io_uring fixes for 6.12-stable. All 9 patches are in Linus's
tree. Patches 3-9 came with the io_uring-7.4 pull, merged as commit
78d9aedf2342 (tag io_uring-7.4-20261119); the others were merged
earlier.
They apply on top of v6.12.118 plus queue-6.12 at stable-queue commit
3dd4b2d3c1f7, and also to v6.12.118 alone.
Patches 1, 3 and 6-7 needed changes for 6.12; the [axboe: ...] note at
the end of each says what was changed and why. The rest are clean
backports.
Patch 1/9 (io_uring: enable toggle of iowait usage when waiting on CQEs)
is a feature, not a fix. It is in stable because the QEMU developers
asked for it: QEMU uses IORING_ENTER_NO_IOWAIT and wants it supported on
all current stable kernels. Nothing changes unless the flag is set.
Testing: ran the liburing-2.15-75-ga10946dc test suite in KVM guests
(PROVE_LOCKING on) on the baseline v6.12.118 kernel and on the patched
kernel, which is v6.12.118 plus these patches and the other io_uring
commits already queued. The patched kernel failed exactly the same tests
as the baseline. The failures that remain are on both kernels (napi-test
timeout, send-zerocopy, timestamp and similar). No new dmesg splat
(WARNING, BUG, lockdep, KASAN, Oops). io_uring/ builds without warnings
on the release plus queue plus patches.
CVE assessment, by the io_uring maintainer:
Should not get a CVE:
1/9 io_uring: enable toggle of iowait usage when waiting on CQEs
Feature, not a fix. IORING_ENTER_NO_IOWAIT only skips setting
in_iowait; nothing changes unless the flag is set.
2/9 io_uring/napi: don't end the single NAPI busy poll early on
need_resched()
The single NAPI busy poll ended on the first need_resched() and
slept for the rest of the timeout. Busy poll duration only, a
performance bug.
3/9 io_uring: fail soft links on negative IOU_COMPLETE results
io_issue_sqe() now calls req_set_fail() for negative IOU_COMPLETE
results, so IOSQE_IO_LINK dependents of a failed request are
cancelled. This only changes link semantics of the caller's own
requests.
4/9 io_uring/futex: fail soft links when cancelling a wait
__io_futex_cancel() now calls req_set_fail(), so linked requests
are cancelled. Only the caller's own link chain is affected.
5/9 io_uring/rw: fail soft links on IOPOLL short or error IO
The IOPOLL completion path now calls req_set_fail() when the
result changed, so linked requests are cancelled after a short or
failed read/write. Only the caller's own link chain is affected.
6/9 io_uring/timeout: don't rearm a multishot timeout on cancel
A cancelled multishot timeout was re-armed from the cancel task
work. A zero interval plus moving the CQ head far ahead only
delays teardown of the caller's own ring; the loop goes through
hrtimer and workqueue, so it reschedules and holds no spinlock.
7/9 io_uring/rsrc: keep the old file when a file update fails
The old file was dropped from the registered slot before the new
fd was checked, so a failed update left the slot empty and posted
its tag CQE. Only the caller's own file table, no refcount or
lifetime problem.
8/9 io_uring/kbuf: initialize arg->out_len from arg->max_len in
io_buffers_select()
Legacy provided buffers ignored sqe->len and used the full
buf->len. The copy is still bounded by the length of the buffer
the app provided, so this is data accounting visible to the app
only.
9/9 io_uring/io-wq: apply the current cpuset to new workers
New io-wq workers took a stale copy of the cpuset, so they could
run on CPUs the cgroup no longer has. cpuset is a CPU placement
control, not a privilege boundary; the worst case is scheduling
interference.
Cheng Yongkang (2):
io_uring/timeout: don't rearm a multishot timeout on cancel
io_uring/rsrc: keep the old file when a file update fails
Cinq Prairie (1):
io_uring/napi: don't end the single NAPI busy poll early on
need_resched()
Jens Axboe (1):
io_uring: enable toggle of iowait usage when waiting on CQEs
Mikhail Malyshev (1):
io_uring/io-wq: apply the current cpuset to new workers
Siddharth Chintamaneni (1):
io_uring/kbuf: initialize arg->out_len from arg->max_len in
io_buffers_select()
Yang Xiuwei (3):
io_uring: fail soft links on negative IOU_COMPLETE results
io_uring/futex: fail soft links when cancelling a wait
io_uring/rw: fail soft links on IOPOLL short or error IO
include/uapi/linux/io_uring.h | 2 ++
io_uring/futex.c | 1 +
io_uring/io-wq.c | 44 +++++++++++++++++++++++++++++++----
io_uring/io_uring.c | 39 +++++++++++++++++++------------
io_uring/io_uring.h | 1 +
io_uring/kbuf.c | 1 +
io_uring/napi.c | 31 +++++++++++-------------
io_uring/rsrc.c | 25 ++++++++++++--------
io_uring/rw.c | 4 +++-
io_uring/timeout.c | 3 ++-
10 files changed, 102 insertions(+), 49 deletions(-)
--
2.55.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 6.12.y 1/9] io_uring: enable toggle of iowait usage when waiting on CQEs
2026-11-21 21:31 [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable Jens Axboe
@ 2026-11-21 21:31 ` Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 2/9] io_uring/napi: don't end the single NAPI busy poll early on need_resched() Jens Axboe
` (7 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2026-11-21 21:31 UTC (permalink / raw)
To: stable
Commit 07754bfd9aee59063f8549f6e4d455eae636ecc7 upstream.
By default, io_uring marks a waiting task as being in iowait, if it's
sleeping waiting on events and there are pending requests. This isn't
necessarily always useful, and may be confusing on non-storage setups
where iowait isn't expected. It can also cause extra power usage, by
preventing the CPU from entering lower sleep states.
This adds a new enter flag, IORING_ENTER_NO_IOWAIT. If set, then
io_uring will not account the sleeping task as being in iowait. If the
kernel supports this feature, then it will be marked by having the
IORING_FEAT_NO_IOWAIT feature flag set.
As the kernel currently does not support separating the iowait
accounting and CPU frequency boosting, the IORING_ENTER_NO_IOWAIT
controls both of these at the same time. In the future, if those do end
up being split, then it'd be possible to control them separately.
However, it seems more likely that the kernel will decouple iowait and
CPU frequency boosting anyway.
[ 6.12: only the NO_IOWAIT flag bits are added, the EXT_ARG_REG and
RW_ATTR bits before them don't exist here. io_get_ext_arg() has no
ctx argument. ]
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
include/uapi/linux/io_uring.h | 2 ++
io_uring/io_uring.c | 31 +++++++++++++++++++------------
2 files changed, 21 insertions(+), 12 deletions(-)
diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
index fdd7b931ea1d..e777e6b128e8 100644
--- a/include/uapi/linux/io_uring.h
+++ b/include/uapi/linux/io_uring.h
@@ -518,6 +518,7 @@ struct io_cqring_offsets {
#define IORING_ENTER_EXT_ARG (1U << 3)
#define IORING_ENTER_REGISTERED_RING (1U << 4)
#define IORING_ENTER_ABS_TIMER (1U << 5)
+#define IORING_ENTER_NO_IOWAIT (1U << 7)
/*
* Passed in for io_uring_setup(2). Copied back with updated info on success
@@ -554,6 +555,7 @@ struct io_uring_params {
#define IORING_FEAT_REG_REG_RING (1U << 13)
#define IORING_FEAT_RECVSEND_BUNDLE (1U << 14)
#define IORING_FEAT_MIN_TIMEOUT (1U << 15)
+#define IORING_FEAT_NO_IOWAIT (1U << 17)
/*
* io_uring_register(2) opcodes and arguments
diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 09551e3907f9..630b306a849f 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -2532,8 +2532,17 @@ static int io_cqring_schedule_timeout(struct io_wait_queue *iowq,
return READ_ONCE(iowq->hit_timeout) ? -ETIME : 0;
}
+struct ext_arg {
+ size_t argsz;
+ struct __kernel_timespec __user *ts;
+ const sigset_t __user *sig;
+ ktime_t min_time;
+ bool iowait;
+};
+
static int __io_cqring_wait_schedule(struct io_ring_ctx *ctx,
struct io_wait_queue *iowq,
+ struct ext_arg *ext_arg,
ktime_t start_time)
{
int ret = 0;
@@ -2543,7 +2552,7 @@ static int __io_cqring_wait_schedule(struct io_ring_ctx *ctx,
* can take into account that the task is waiting for IO - turns out
* to be important for low QD IO.
*/
- if (current_pending_io())
+ if (ext_arg->iowait && current_pending_io())
current->in_iowait = 1;
if (iowq->timeout != KTIME_MAX || iowq->min_timeout)
ret = io_cqring_schedule_timeout(iowq, ctx->clockid, start_time);
@@ -2556,6 +2565,7 @@ static int __io_cqring_wait_schedule(struct io_ring_ctx *ctx,
/* If this returns > 0, the caller should retry */
static inline int io_cqring_wait_schedule(struct io_ring_ctx *ctx,
struct io_wait_queue *iowq,
+ struct ext_arg *ext_arg,
ktime_t start_time)
{
if (unlikely(READ_ONCE(ctx->check_cq)))
@@ -2569,16 +2579,9 @@ static inline int io_cqring_wait_schedule(struct io_ring_ctx *ctx,
if (unlikely(io_should_wake(iowq)))
return 0;
- return __io_cqring_wait_schedule(ctx, iowq, start_time);
+ return __io_cqring_wait_schedule(ctx, iowq, ext_arg, start_time);
}
-struct ext_arg {
- size_t argsz;
- struct __kernel_timespec __user *ts;
- const sigset_t __user *sig;
- ktime_t min_time;
-};
-
/*
* Wait until events become available, if we don't already have some. The
* application must reap them itself, as they reside on the shared cq ring.
@@ -2660,7 +2663,7 @@ static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events, u32 flags,
TASK_INTERRUPTIBLE);
}
- ret = io_cqring_wait_schedule(ctx, &iowq, start_time);
+ ret = io_cqring_wait_schedule(ctx, &iowq, ext_arg, start_time);
__set_current_state(TASK_RUNNING);
atomic_set(&ctx->cq_wait_nr, IO_CQ_WAKE_INIT);
@@ -3363,6 +3366,8 @@ static int io_get_ext_arg(unsigned flags, const void __user *argp,
{
struct io_uring_getevents_arg arg;
+ ext_arg->iowait = !(flags & IORING_ENTER_NO_IOWAIT);
+
/*
* If EXT_ARG isn't set, then we have no timespec and the argp pointer
* is just a pointer to the sigset_t.
@@ -3399,7 +3404,8 @@ SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit,
if (unlikely(flags & ~(IORING_ENTER_GETEVENTS | IORING_ENTER_SQ_WAKEUP |
IORING_ENTER_SQ_WAIT | IORING_ENTER_EXT_ARG |
IORING_ENTER_REGISTERED_RING |
- IORING_ENTER_ABS_TIMER)))
+ IORING_ENTER_ABS_TIMER |
+ IORING_ENTER_NO_IOWAIT)))
return -EINVAL;
/*
@@ -3786,7 +3792,8 @@ static __cold int io_uring_create(unsigned entries, struct io_uring_params *p,
IORING_FEAT_EXT_ARG | IORING_FEAT_NATIVE_WORKERS |
IORING_FEAT_RSRC_TAGS | IORING_FEAT_CQE_SKIP |
IORING_FEAT_LINKED_FILE | IORING_FEAT_REG_REG_RING |
- IORING_FEAT_RECVSEND_BUNDLE | IORING_FEAT_MIN_TIMEOUT;
+ IORING_FEAT_RECVSEND_BUNDLE | IORING_FEAT_MIN_TIMEOUT |
+ IORING_FEAT_NO_IOWAIT;
if (copy_to_user(params, p, sizeof(*p))) {
ret = -EFAULT;
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 6.12.y 2/9] io_uring/napi: don't end the single NAPI busy poll early on need_resched()
2026-11-21 21:31 [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 1/9] io_uring: enable toggle of iowait usage when waiting on CQEs Jens Axboe
@ 2026-11-21 21:31 ` Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 3/9] io_uring: fail soft links on negative IOU_COMPLETE results Jens Axboe
` (6 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2026-11-21 21:31 UTC (permalink / raw)
To: stable
From: Cinq Prairie <notprairie@gmail.com>
Commit 39c7210e970200f8b17de3767c73efe64173a95f upstream.
With a single registered NAPI, io_napi_blocking_busy_loop() hands
io_napi_busy_loop_should_end() to napi_busy_loop_rcu() as loop_end and
runs its own loop once, relying on __napi_busy_loop() to spin until
that predicate fires. napi_busy_loop_rcu() sets NAPI_F_END_ON_RESCHED,
so __napi_busy_loop() also returns on the first need_resched(). io_uring
treats the return as the end of the busy poll period and sleeps in
schedule_hrtimeout() for the rest of the wait timeout.
With two or more NAPIs the inner loop is single-shot and io_uring's own
loop runs until its predicate is met, so busy_poll_to is honoured. The
same registration therefore behaves differently for one NAPI than for
several.
Make the busy poll deadline absolute, stored in io_wait_queue, so that
io_napi_busy_loop_should_end() no longer depends on the start_time the
napi loop snapshots on every entry. The single NAPI loop can then be
re-entered after an early need_resched() exit if there is budget left,
and the loop condition drops the "run once" special case. The fast path
that holds the NAPI across polls is kept.
Measured on io_uring-7.4 at 9c207aa58e54, x86-64, HZ=1000,
PREEMPT_DYNAMIC (lazy), no CPU isolation, low RX traffic, r8169 with one
RX queue, prefer_busy_poll, busy_poll_to = wait timeout = 10 ms.
"waker" is a SCHED_FIFO task pinned to the polling CPU waking every 1 ms:
1 NAPI, waker, before: busy 0.5-1.5 ms per 10 ms wait, ~0.8M polls/s
1 NAPI, waker, after: busy 10 ms per 10 ms wait, ~16.2M polls/s
1 NAPI, idle, before: busy 10 ms per 10 ms wait, ~15.3M polls/s
1 NAPI, idle, after: busy 10 ms per 10 ms wait, ~16.2M polls/s
2 NAPIs, waker, before: busy 10 ms per 10 ms wait, ~7.2M polls/s
2 NAPIs, waker, after: busy 10 ms per 10 ms wait, ~7.1M polls/s
The held-NAPI fast path polls about twice as often as the single-shot
path used for several NAPIs, which is why it is kept.
Fixes: 8d0c12a80cde ("io-uring: add napi busy poll support")
Cc: stable@vger.kernel.org
Signed-off-by: Cinq Prairie <notprairie@gmail.com>
Link: https://patch.msgid.link/20261027204840.7823-1-notprairie@gmail.com
[axboe: trim comments, drop __always_unused]
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
io_uring/io_uring.h | 1 +
io_uring/napi.c | 31 ++++++++++++++-----------------
2 files changed, 15 insertions(+), 17 deletions(-)
diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h
index 628ced4b44b2..1cdfc5725e01 100644
--- a/io_uring/io_uring.h
+++ b/io_uring/io_uring.h
@@ -48,6 +48,7 @@ struct io_wait_queue {
#ifdef CONFIG_NET_RX_BUSY_POLL
ktime_t napi_busy_poll_dt;
+ ktime_t napi_busy_poll_end;
bool napi_prefer_busy_poll;
#endif
};
diff --git a/io_uring/napi.c b/io_uring/napi.c
index bdae13367e0f..ade35aff6375 100644
--- a/io_uring/napi.c
+++ b/io_uring/napi.c
@@ -107,19 +107,15 @@ static inline void io_napi_remove_stale(struct io_ring_ctx *ctx, bool is_stale)
__io_napi_remove_stale(ctx);
}
-static inline bool io_napi_busy_loop_timeout(ktime_t start_time,
- ktime_t bp)
+static inline bool io_napi_busy_loop_timeout(struct io_wait_queue *iowq)
{
- if (bp) {
- ktime_t end_time = ktime_add(start_time, bp);
- ktime_t now = net_to_ktime(busy_loop_current_time());
-
- return ktime_after(now, end_time);
- }
-
- return true;
+ if (!iowq->napi_busy_poll_dt)
+ return true;
+ return ktime_after(net_to_ktime(busy_loop_current_time()),
+ iowq->napi_busy_poll_end);
}
+/* Uses an absolute deadline, so the napi loop can be re-entered */
static bool io_napi_busy_loop_should_end(void *data,
unsigned long start_time)
{
@@ -129,8 +125,7 @@ static bool io_napi_busy_loop_should_end(void *data,
return true;
if (io_should_wake(iowq) || io_has_work(iowq->ctx))
return true;
- if (io_napi_busy_loop_timeout(net_to_ktime(start_time),
- iowq->napi_busy_poll_dt))
+ if (io_napi_busy_loop_timeout(iowq))
return true;
return false;
@@ -157,13 +152,16 @@ static bool __io_napi_do_busy_loop(struct io_ring_ctx *ctx,
static void io_napi_blocking_busy_loop(struct io_ring_ctx *ctx,
struct io_wait_queue *iowq)
{
- unsigned long start_time = busy_loop_current_time();
+ ktime_t now = net_to_ktime(busy_loop_current_time());
bool (*loop_end)(void *, unsigned long) = NULL;
void *loop_end_arg = NULL;
bool is_stale = false;
- /* Singular lists use a different napi loop end check function and are
- * only executed once.
+ iowq->napi_busy_poll_end = ktime_add(now, iowq->napi_busy_poll_dt);
+
+ /*
+ * A single NAPI is held by __napi_busy_loop() for the whole spin. It
+ * may return early on need_resched(), loop until our own deadline.
*/
if (list_is_singular(&ctx->napi_list)) {
loop_end = io_napi_busy_loop_should_end;
@@ -174,8 +172,7 @@ static void io_napi_blocking_busy_loop(struct io_ring_ctx *ctx,
do {
is_stale = __io_napi_do_busy_loop(ctx, loop_end,
loop_end_arg);
- } while (!io_napi_busy_loop_should_end(iowq, start_time) &&
- !loop_end_arg);
+ } while (!io_napi_busy_loop_should_end(iowq, 0));
}
io_napi_remove_stale(ctx, is_stale);
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 6.12.y 3/9] io_uring: fail soft links on negative IOU_COMPLETE results
2026-11-21 21:31 [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 1/9] io_uring: enable toggle of iowait usage when waiting on CQEs Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 2/9] io_uring/napi: don't end the single NAPI busy poll early on need_resched() Jens Axboe
@ 2026-11-21 21:31 ` Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 4/9] io_uring/futex: fail soft links when cancelling a wait Jens Axboe
` (5 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2026-11-21 21:31 UTC (permalink / raw)
To: stable
From: Yang Xiuwei <yangxiuwei@kylinos.cn>
Commit 9ed19503a7b34e1ffb2b81236843204c473b0ae0 upstream.
Handlers such as fsync return IOU_COMPLETE with a negative cqe.res.
The old io_req_complete() path called req_set_fail() for res < 0.
After completions moved into the core, that check was dropped,
so IOSQE_IO_LINK dependents kept running after the head failed.
Fixes: 97b388d70b53 ("io_uring: handle completions in the core")
Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn>
Link: https://patch.msgid.link/20261114041123.109163-1-yangxiuwei@kylinos.cn
Signed-off-by: Jens Axboe <axboe@kernel.dk>
[axboe: 6.12 has IOU_OK rather than IOU_COMPLETE]
---
io_uring/io_uring.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 630b306a849f..3f8a0f7e84f6 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -1789,9 +1789,11 @@ static int io_issue_sqe(struct io_kiocb *req, unsigned int issue_flags)
}
if (ret == IOU_OK) {
- /* not all handlers take care of the internal restart codes */
- if (unlikely(req->cqe.res <= -ERESTARTSYS))
- req->cqe.res = io_fixup_restart_res(req->cqe.res);
+ if (unlikely(req->cqe.res < 0)) {
+ if (req->cqe.res <= -ERESTARTSYS)
+ req->cqe.res = io_fixup_restart_res(req->cqe.res);
+ req_set_fail(req);
+ }
if (issue_flags & IO_URING_F_COMPLETE_DEFER)
io_req_complete_defer(req);
else
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 6.12.y 4/9] io_uring/futex: fail soft links when cancelling a wait
2026-11-21 21:31 [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable Jens Axboe
` (2 preceding siblings ...)
2026-11-21 21:31 ` [PATCH 6.12.y 3/9] io_uring: fail soft links on negative IOU_COMPLETE results Jens Axboe
@ 2026-11-21 21:31 ` Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 5/9] io_uring/rw: fail soft links on IOPOLL short or error IO Jens Axboe
` (4 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2026-11-21 21:31 UTC (permalink / raw)
To: stable
From: Yang Xiuwei <yangxiuwei@kylinos.cn>
Commit 9a7cbe50dd95594d6fb3c57b8454ace58d8d8d0a upstream.
__io_futex_cancel() completes the request with -ECANCELED but does
not call req_set_fail(). IOSQE_IO_LINK dependents then still run.
Fixes: 194bb58c6090 ("io_uring: add support for futex wake and wait")
Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn>
Link: https://patch.msgid.link/20261114072246.741950-2-yangxiuwei@kylinos.cn
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
io_uring/futex.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/io_uring/futex.c b/io_uring/futex.c
index 990568baf393..59b679e1e26f 100644
--- a/io_uring/futex.c
+++ b/io_uring/futex.c
@@ -108,6 +108,7 @@ static bool __io_futex_cancel(struct io_ring_ctx *ctx, struct io_kiocb *req)
}
hlist_del_init(&req->hash_node);
+ req_set_fail(req);
io_req_set_res(req, -ECANCELED, 0);
io_req_task_work_add(req);
return true;
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 6.12.y 5/9] io_uring/rw: fail soft links on IOPOLL short or error IO
2026-11-21 21:31 [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable Jens Axboe
` (3 preceding siblings ...)
2026-11-21 21:31 ` [PATCH 6.12.y 4/9] io_uring/futex: fail soft links when cancelling a wait Jens Axboe
@ 2026-11-21 21:31 ` Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 6/9] io_uring/timeout: don't rearm a multishot timeout on cancel Jens Axboe
` (3 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2026-11-21 21:31 UTC (permalink / raw)
To: stable
From: Yang Xiuwei <yangxiuwei@kylinos.cn>
Commit 4d3db66c5363dcf42c3d43ebaea330ac2d1888bb upstream.
io_complete_rw_iopoll() updates cqe.res when the transfer does not
match the requested length, but it does not call req_set_fail().
Soft linked requests then still run after a short or failed IOPOLL
read or write.
Fixes: b66ceaf324b3 ("io_uring: move iopoll reissue into regular IO path")
Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn>
Link: https://patch.msgid.link/20261114072246.741950-3-yangxiuwei@kylinos.cn
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
io_uring/rw.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/io_uring/rw.c b/io_uring/rw.c
index b18b10cdbb52..688ce923f6a3 100644
--- a/io_uring/rw.c
+++ b/io_uring/rw.c
@@ -553,8 +553,10 @@ static void io_complete_rw_iopoll(struct kiocb *kiocb, long res)
if (res == -EAGAIN && io_rw_should_reissue(req))
req->flags |= REQ_F_REISSUE | REQ_F_BL_NO_RECYCLE;
- else if (unlikely(final_res != req->cqe.res))
+ else if (unlikely(final_res != req->cqe.res)) {
+ req_set_fail(req);
req->cqe.res = final_res;
+ }
/* order with io_iopoll_complete() checking ->iopoll_completed */
smp_store_release(&req->iopoll_completed, 1);
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 6.12.y 6/9] io_uring/timeout: don't rearm a multishot timeout on cancel
2026-11-21 21:31 [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable Jens Axboe
` (4 preceding siblings ...)
2026-11-21 21:31 ` [PATCH 6.12.y 5/9] io_uring/rw: fail soft links on IOPOLL short or error IO Jens Axboe
@ 2026-11-21 21:31 ` Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 7/9] io_uring/rsrc: keep the old file when a file update fails Jens Axboe
` (2 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2026-11-21 21:31 UTC (permalink / raw)
To: stable
From: Cheng Yongkang <teel4res@gmail.com>
Commit 97d4a585db2b91d855f957be0a9f126068c5be93 upstream.
io_timeout_complete() does not check tw.cancel. When it runs from
io_cancel_local_task_work(), a multishot timeout that has not reached
its count still posts an IORING_CQE_F_MORE completion and rearms its
timer. With a zero interval the timer expires again right away and
queues new task work, so the drain keeps running it until posting a
CQE fails.
Normally the CQ fills up quickly and the request then completes. But
if userspace moves the CQ head far ahead of the tail, for example to
cq.tail + 0x7fffffff, posting keeps succeeding and ring teardown is
delayed for a long time.
Don't post another CQE or rearm the timer when tw.cancel is set, so
the request completes through io_req_task_complete(), like the poll and
linked timeout task work does when cancelled.
Fixes: ea97f6c8558e ("io_uring: add support for multishot timeouts")
Signed-off-by: Cheng Yongkang <teel4res@gmail.com>
Link: https://patch.msgid.link/20261116044519.281756-1-teel4res@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
[axboe: 6.12 has no tw.cancel, use io_should_terminate_tw() like
io_req_task_link_timeout() does]
---
io_uring/timeout.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/io_uring/timeout.c b/io_uring/timeout.c
index d71b0fdbb066..e99709c4089d 100644
--- a/io_uring/timeout.c
+++ b/io_uring/timeout.c
@@ -72,7 +72,8 @@ static void io_timeout_complete(struct io_kiocb *req, struct io_tw_state *ts)
struct io_ring_ctx *ctx = req->ctx;
if (!io_timeout_finish(timeout, data)) {
- if (io_req_post_cqe(req, -ETIME, IORING_CQE_F_MORE)) {
+ if (!io_should_terminate_tw(ctx) &&
+ io_req_post_cqe(req, -ETIME, IORING_CQE_F_MORE)) {
/* re-arm timer */
spin_lock_irq(&ctx->timeout_lock);
list_add(&timeout->list, ctx->timeout_list.prev);
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 6.12.y 7/9] io_uring/rsrc: keep the old file when a file update fails
2026-11-21 21:31 [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable Jens Axboe
` (5 preceding siblings ...)
2026-11-21 21:31 ` [PATCH 6.12.y 6/9] io_uring/timeout: don't rearm a multishot timeout on cancel Jens Axboe
@ 2026-11-21 21:31 ` Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 8/9] io_uring/kbuf: initialize arg->out_len from arg->max_len in io_buffers_select() Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 9/9] io_uring/io-wq: apply the current cpuset to new workers Jens Axboe
8 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2026-11-21 21:31 UTC (permalink / raw)
To: stable
From: Cheng Yongkang <teel4res@gmail.com>
Commit e5a346e4309d720c47fe8331a00068608fad47dd upstream.
__io_sqe_files_update() resets the slot before it calls fget() on the
new fd. If fget() fails, the fd is an io_uring file, or the node
allocation fails, the update stops and reports that entry as not done,
but the old file has already been removed. Later requests that use the
slot fail with -EBADF, and if the old file was registered with a tag,
its tag CQE is posted as if it had been removed on purpose.
Reset the slot only after the new file has been checked and its node
allocated, as io_install_fixed_file() does. Removal with fd == -1 still
resets the slot right away.
Fixes: c3a31e605620 ("io_uring: add support for IORING_REGISTER_FILES_UPDATE")
Cc: stable@vger.kernel.org
Signed-off-by: Cheng Yongkang <teel4res@gmail.com>
Link: https://patch.msgid.link/20261116044531.281978-1-teel4res@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
[axboe: 6.12 has no rsrc nodes for files, the old file is queued for
removal with io_queue_rsrc_removal(), which can fail. Get and check the
new file first, and only remove the old one once that has succeeded]
---
io_uring/rsrc.c | 25 +++++++++++++++----------
1 file changed, 15 insertions(+), 10 deletions(-)
diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index 168c6987435e..5a3f3e4e3fa1 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -348,6 +348,7 @@ static int __io_sqe_files_update(struct io_ring_ctx *ctx,
return -EINVAL;
for (done = 0; done < nr_args; done++) {
+ struct file *file = NULL;
u64 tag = 0;
if ((tags && copy_from_user(&tag, &tags[done], sizeof(tag))) ||
@@ -365,17 +366,8 @@ static int __io_sqe_files_update(struct io_ring_ctx *ctx,
i = array_index_nospec(up->offset + done, ctx->nr_user_files);
file_slot = io_fixed_file_slot(&ctx->file_table, i);
- if (file_slot->file_ptr) {
- err = io_queue_rsrc_removal(data, i,
- io_slot_file(file_slot));
- if (err)
- break;
- file_slot->file_ptr = 0;
- io_file_bitmap_clear(&ctx->file_table, i);
- }
if (fd != -1) {
- struct file *file = fget(fd);
-
+ file = fget(fd);
if (!file) {
err = -EBADF;
break;
@@ -388,6 +380,19 @@ static int __io_sqe_files_update(struct io_ring_ctx *ctx,
err = -EBADF;
break;
}
+ }
+ if (file_slot->file_ptr) {
+ err = io_queue_rsrc_removal(data, i,
+ io_slot_file(file_slot));
+ if (err) {
+ if (file)
+ fput(file);
+ break;
+ }
+ file_slot->file_ptr = 0;
+ io_file_bitmap_clear(&ctx->file_table, i);
+ }
+ if (file) {
*io_get_tag_slot(data, i) = tag;
io_fixed_file_set(file_slot, file);
io_file_bitmap_set(&ctx->file_table, i);
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 6.12.y 8/9] io_uring/kbuf: initialize arg->out_len from arg->max_len in io_buffers_select()
2026-11-21 21:31 [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable Jens Axboe
` (6 preceding siblings ...)
2026-11-21 21:31 ` [PATCH 6.12.y 7/9] io_uring/rsrc: keep the old file when a file update fails Jens Axboe
@ 2026-11-21 21:31 ` Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 9/9] io_uring/io-wq: apply the current cpuset to new workers Jens Axboe
8 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2026-11-21 21:31 UTC (permalink / raw)
To: stable
From: Siddharth Chintamaneni <sidchintamaneni@gmail.com>
Commit b7322810d1870ccc0d293ad383d94ef583d5057b upstream.
io_buffers_select() passes &arg->out_len to io_provided_buffers_select()
on the legacy provided-buffer path. Because callers initialize
arg->max_len (from sqe->len) while leaving arg->out_len as 0,
io_provided_buffers_select() sees *len == 0 and always selects the full
buffer size buf->len, ignoring the user-specified sqe->len limit.
Initialize arg->out_len = arg->max_len before calling
io_provided_buffers_select().
Fixes: 35c8711c8fc4 ("io_uring/kbuf: add helpers for getting/peeking multiple buffers")
Signed-off-by: Siddharth Chintamaneni <sidchintamaneni@gmail.com>
Link: https://patch.msgid.link/20261116193146.65315-1-sidchintamaneni@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
io_uring/kbuf.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/io_uring/kbuf.c b/io_uring/kbuf.c
index da6b46244e6d..9d6b7c433eea 100644
--- a/io_uring/kbuf.c
+++ b/io_uring/kbuf.c
@@ -343,6 +343,7 @@ int io_buffers_select(struct io_kiocb *req, struct buf_sel_arg *arg,
req->flags |= REQ_F_BUF_MORE;
}
} else {
+ arg->out_len = arg->max_len;
ret = io_provided_buffers_select(req, &arg->out_len, sel->buf_list, arg->iovs);
}
out_unlock:
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 6.12.y 9/9] io_uring/io-wq: apply the current cpuset to new workers
2026-11-21 21:31 [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable Jens Axboe
` (7 preceding siblings ...)
2026-11-21 21:31 ` [PATCH 6.12.y 8/9] io_uring/kbuf: initialize arg->out_len from arg->max_len in io_buffers_select() Jens Axboe
@ 2026-11-21 21:31 ` Jens Axboe
8 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2026-11-21 21:31 UTC (permalink / raw)
To: stable
From: Mikhail Malyshev <mike.malyshev@gmail.com>
Commit 0659e44ce739ecd856164c75a5325b7959a2921d upstream.
Since commit 84eacf177faa ("io_uring/io-wq: inherit cpuset of cgroup in
io worker"), io_wq_create() copies the cpuset of the creating task into
wq->cpu_mask, and io_init_new_worker() gives each new worker that copy.
A cpuset change, or a move of the task to another cgroup, updates the
workers that exist but not the copy, so workers started later run on
the CPUs the cgroup had when the io-wq was created, which may since
have been given to other tasks. Workers come and go for the life of the
io-wq, and a worker that cpuset moved is also retired at its next idle
timeout and replaced on the old CPUs.
Keep wq->cpu_mask for the affinity of the io-wq itself, all CPUs or the
mask set with IORING_REGISTER_IOWQ_AFF, and intersect it with the cpuset
of the worker when the worker starts, falling back to the cpuset if the
two do not intersect, as __set_cpus_allowed_ptr() does. Use the same
mask in the idle exit test, and go back to all CPUs when the registered
affinity is removed.
Fixes: 84eacf177faa ("io_uring/io-wq: inherit cpuset of cgroup in io worker")
Cc: stable@vger.kernel.org
Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
Link: https://patch.msgid.link/20261111162705.1150983-1-mike.malyshev@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
io_uring/io-wq.c | 44 +++++++++++++++++++++++++++++++++++++++-----
1 file changed, 39 insertions(+), 5 deletions(-)
diff --git a/io_uring/io-wq.c b/io_uring/io-wq.c
index 181eecdb32c0..91b83751795f 100644
--- a/io_uring/io-wq.c
+++ b/io_uring/io-wq.c
@@ -687,6 +687,34 @@ static bool io_worker_retire(struct io_wq_acct *acct, struct io_worker *worker)
return false;
}
+/*
+ * The CPUs a worker of @wq runs on: those of the io-wq affinity that the
+ * worker's cpuset allows, or the cpuset if they have none in common, as the
+ * scheduler does with a task's requested affinity when its cpuset changes.
+ * Read the cpuset each time: it can change while the io-wq exists.
+ */
+static void io_wq_worker_cpus(struct io_wq *wq, struct task_struct *tsk,
+ struct cpumask *cpus)
+{
+ cpuset_cpus_allowed(tsk, cpus);
+ if (cpumask_intersects(cpus, wq->cpu_mask))
+ cpumask_and(cpus, cpus, wq->cpu_mask);
+}
+
+/* Is the current worker on a CPU that a new worker would not get? */
+static bool io_wq_worker_off_cpus(struct io_wq *wq)
+{
+ cpumask_var_t cpus;
+ bool off;
+
+ if (!alloc_cpumask_var(&cpus, GFP_KERNEL))
+ return false;
+ io_wq_worker_cpus(wq, current, cpus);
+ off = !cpumask_test_cpu(raw_smp_processor_id(), cpus);
+ free_cpumask_var(cpus);
+ return off;
+}
+
static int io_wq_worker(void *data)
{
struct io_worker *worker = data;
@@ -748,8 +776,7 @@ static int io_wq_worker(void *data)
}
if (!ret) {
last_timeout = true;
- exit_mask = !cpumask_test_cpu(raw_smp_processor_id(),
- wq->cpu_mask);
+ exit_mask = io_wq_worker_off_cpus(wq);
}
}
@@ -799,9 +826,16 @@ void io_wq_worker_sleeping(struct task_struct *tsk)
static void io_init_new_worker(struct io_wq *wq, struct io_worker *worker,
struct task_struct *tsk)
{
+ cpumask_var_t cpus;
+
tsk->worker_private = worker;
worker->task = tsk;
- set_cpus_allowed_ptr(tsk, wq->cpu_mask);
+ /* Without a mask, the worker keeps its creator's CPUs. */
+ if (alloc_cpumask_var(&cpus, GFP_KERNEL)) {
+ io_wq_worker_cpus(wq, tsk, cpus);
+ set_cpus_allowed_ptr(tsk, cpus);
+ free_cpumask_var(cpus);
+ }
raw_spin_lock(&wq->lock);
hlist_nulls_add_head_rcu(&worker->nulls_node, &wq->free_list);
@@ -1269,7 +1303,7 @@ struct io_wq *io_wq_create(unsigned bounded, struct io_wq_data *data)
if (!alloc_cpumask_var(&wq->cpu_mask, GFP_KERNEL))
goto err;
- cpuset_cpus_allowed(data->task, wq->cpu_mask);
+ cpumask_copy(wq->cpu_mask, cpu_possible_mask);
wq->acct[IO_WQ_ACCT_BOUND].max_workers = bounded;
wq->acct[IO_WQ_ACCT_UNBOUND].max_workers =
task_rlimit(current, RLIMIT_NPROC);
@@ -1453,7 +1487,7 @@ int io_wq_cpu_affinity(struct io_uring_task *tctx, cpumask_var_t mask)
else
ret = -EINVAL;
} else {
- cpumask_copy(tctx->io_wq->cpu_mask, allowed_mask);
+ cpumask_copy(tctx->io_wq->cpu_mask, cpu_possible_mask);
}
rcu_read_unlock();
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-11-21 21:31 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-21 21:31 [PATCH 6.12.y 0/9] io_uring fixes for 6.12-stable Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 1/9] io_uring: enable toggle of iowait usage when waiting on CQEs Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 2/9] io_uring/napi: don't end the single NAPI busy poll early on need_resched() Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 3/9] io_uring: fail soft links on negative IOU_COMPLETE results Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 4/9] io_uring/futex: fail soft links when cancelling a wait Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 5/9] io_uring/rw: fail soft links on IOPOLL short or error IO Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 6/9] io_uring/timeout: don't rearm a multishot timeout on cancel Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 7/9] io_uring/rsrc: keep the old file when a file update fails Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 8/9] io_uring/kbuf: initialize arg->out_len from arg->max_len in io_buffers_select() Jens Axboe
2026-11-21 21:31 ` [PATCH 6.12.y 9/9] io_uring/io-wq: apply the current cpuset to new workers Jens Axboe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox