From: Jens Axboe <axboe@kernel.dk>
To: stable@vger.kernel.org
Subject: [PATCH 6.6.y 01/19] io_uring: enable toggle of iowait usage when waiting on CQEs
Date: Sat, 21 Nov 2026 21:30:31 +0000 [thread overview]
Message-ID: <20261121213049.1788332-2-axboe@kernel.dk> (raw)
In-Reply-To: <20261121213049.1788332-1-axboe@kernel.dk>
Commit 07754bfd9aee59063f8549f6e4d455eae636ecc7 upstream.
By default, io_uring marks a waiting task as being in iowait, if it's
sleeping waiting on events and there are pending requests. This isn't
necessarily always useful, and may be confusing on non-storage setups
where iowait isn't expected. It can also cause extra power usage, by
preventing the CPU from entering lower sleep states.
This adds a new enter flag, IORING_ENTER_NO_IOWAIT. If set, then
io_uring will not account the sleeping task as being in iowait. If the
kernel supports this feature, then it will be marked by having the
IORING_FEAT_NO_IOWAIT feature flag set.
As the kernel currently does not support separating the iowait
accounting and CPU frequency boosting, the IORING_ENTER_NO_IOWAIT
controls both of these at the same time. In the future, if those do end
up being split, then it'd be possible to control them separately.
However, it seems more likely that the kernel will decouple iowait and
CPU frequency boosting anyway.
[ 6.6: no struct ext_arg here. Pass the enter flags to io_cqring_wait()
and keep the setting in struct io_wait_queue. Flag bits match
upstream. ]
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
include/uapi/linux/io_uring.h | 2 ++
io_uring/io_uring.c | 16 ++++++++++------
2 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
index 333769bc6abf..7c8c70859c94 100644
--- a/include/uapi/linux/io_uring.h
+++ b/include/uapi/linux/io_uring.h
@@ -466,6 +466,7 @@ struct io_cqring_offsets {
#define IORING_ENTER_SQ_WAIT (1U << 2)
#define IORING_ENTER_EXT_ARG (1U << 3)
#define IORING_ENTER_REGISTERED_RING (1U << 4)
+#define IORING_ENTER_NO_IOWAIT (1U << 7)
/*
* Passed in for io_uring_setup(2). Copied back with updated info on success
@@ -500,6 +501,7 @@ struct io_uring_params {
#define IORING_FEAT_CQE_SKIP (1U << 11)
#define IORING_FEAT_LINKED_FILE (1U << 12)
#define IORING_FEAT_REG_REG_RING (1U << 13)
+#define IORING_FEAT_NO_IOWAIT (1U << 17)
/*
* io_uring_register(2) opcodes and arguments
diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 43860048d2bf..b14651c187d4 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -2520,6 +2520,7 @@ struct io_wait_queue {
unsigned cq_tail;
unsigned nr_timeouts;
ktime_t timeout;
+ bool iowait;
};
static inline bool io_has_work(struct io_ring_ctx *ctx)
@@ -2601,7 +2602,7 @@ static inline int io_cqring_wait_schedule(struct io_ring_ctx *ctx,
* can take into account that the task is waiting for IO - turns out
* to be important for low QD IO.
*/
- if (current_pending_io())
+ if (iowq->iowait && current_pending_io())
current->in_iowait = 1;
ret = 0;
if (iowq->timeout == KTIME_MAX)
@@ -2616,7 +2617,7 @@ static inline int io_cqring_wait_schedule(struct io_ring_ctx *ctx,
* Wait until events become available, if we don't already have some. The
* application must reap them itself, as they reside on the shared cq ring.
*/
-static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events,
+static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events, u32 flags,
const sigset_t __user *sig, size_t sigsz,
struct __kernel_timespec __user *uts)
{
@@ -2638,6 +2639,7 @@ static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events,
iowq.wq.private = current;
INIT_LIST_HEAD(&iowq.wq.entry);
iowq.ctx = ctx;
+ iowq.iowait = !(flags & IORING_ENTER_NO_IOWAIT);
iowq.nr_timeouts = atomic_read(&ctx->cq_timeouts);
iowq.cq_tail = READ_ONCE(ctx->rings->cq.head) + min_events;
iowq.timeout = KTIME_MAX;
@@ -3828,7 +3830,8 @@ SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit,
if (unlikely(flags & ~(IORING_ENTER_GETEVENTS | IORING_ENTER_SQ_WAKEUP |
IORING_ENTER_SQ_WAIT | IORING_ENTER_EXT_ARG |
- IORING_ENTER_REGISTERED_RING)))
+ IORING_ENTER_REGISTERED_RING |
+ IORING_ENTER_NO_IOWAIT)))
return -EINVAL;
/*
@@ -3928,8 +3931,8 @@ SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit,
if (likely(!ret2)) {
min_complete = min(min_complete,
ctx->cq_entries);
- ret2 = io_cqring_wait(ctx, min_complete, sig,
- argsz, ts);
+ ret2 = io_cqring_wait(ctx, min_complete, flags,
+ sig, argsz, ts);
}
}
@@ -4211,7 +4214,8 @@ static __cold int io_uring_create(unsigned entries, struct io_uring_params *p,
IORING_FEAT_POLL_32BITS | IORING_FEAT_SQPOLL_NONFIXED |
IORING_FEAT_EXT_ARG | IORING_FEAT_NATIVE_WORKERS |
IORING_FEAT_RSRC_TAGS | IORING_FEAT_CQE_SKIP |
- IORING_FEAT_LINKED_FILE | IORING_FEAT_REG_REG_RING;
+ IORING_FEAT_LINKED_FILE | IORING_FEAT_REG_REG_RING |
+ IORING_FEAT_NO_IOWAIT;
if (copy_to_user(params, p, sizeof(*p))) {
ret = -EFAULT;
--
2.55.0
next prev parent reply other threads:[~2026-11-21 21:30 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-21 21:30 [PATCH 6.6.y 00/19] io_uring fixes for 6.6-stable Jens Axboe
2026-11-21 21:30 ` Jens Axboe [this message]
2026-11-21 21:30 ` [PATCH 6.6.y 02/19] io_uring: fail soft links on negative IOU_COMPLETE results Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 03/19] io_uring/rw: fail soft links on IOPOLL short or error IO Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 04/19] io_uring/timeout: don't rearm a multishot timeout on cancel Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 05/19] io_uring/rsrc: keep the old file when a file update fails Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 06/19] io_uring: cleanup handle_tw_list() calling convention Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 07/19] io_uring: pass in counter to handle_tw_list() rather than return it Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 08/19] io_uring/sqpoll: manage task_work privately Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 09/19] io_uring: wake SQPOLL task when task_work is added to an empty queue Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 10/19] io_uring/sqpoll: use the correct check for pending task_work Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 11/19] io_uring/sqpoll: early exit thread if task_context wasn't allocated Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 12/19] io_uring/sqpoll: ensure that normal task_work is also run timely Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 13/19] io_uring: signal SQPOLL task_work with TWA_SIGNAL_NO_IPI Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 14/19] io_uring/sqpoll: ensure task state is TASK_RUNNING when running task_work Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 15/19] io_uring: don't touch sqd->thread off tw add Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 16/19] io_uring/sqpoll: zero sqd->thread on tctx errors Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 17/19] io_uring/sqpoll: Increase task_work submission batch size Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 18/19] io_uring: fix task_work add use-after-free with SQPOLL Jens Axboe
2026-11-21 21:30 ` [PATCH 6.6.y 19/19] io_uring/sqpoll: do the return-to-userspace work in the SQPOLL thread Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261121213049.1788332-2-axboe@kernel.dk \
--to=axboe@kernel.dk \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox