All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: Excessive IO PSI for iothread when using io_uring since QEMU 10.2
@ 2026-11-15 23:05 John Maguire
  2026-11-18 14:06 ` Fernando Casas Schösow
  2026-11-18 15:58 ` Jens Axboe
  0 siblings, 2 replies; 17+ messages in thread
From: John Maguire @ 2026-11-15 23:05 UTC (permalink / raw)
  To: stefanha
  Cc: axboe, f.ebner, fam, hreitz, kwolf, qemu-block, qemu-devel,
	t.lamprecht

On Tue, Jun 02, 2026, Stefan Hajnoczi wrote:
> Pinging Jens: io_uring accounts POLL_ADD SQEs as blocking I/O activity
> whereas select(2)/poll(2)/epoll_wait(2) do not. Would it make sense to
> follow the same accounting as the syscalls for this operation since that
> is probably expected?

Hello,

I run Incus on two hosts and see misleading iowait and IO pressure on
both since Incus 7.6, which puts virtio disks on a dedicated iothread by
default. Incus users now hit this without opting into iothreads.

Host 1: Arch Linux, stock QEMU 11.1.3, kernel 6.18.62, 16 CPUs, five
VMs. /proc/pressure/io shows about 80% "some" and 75% "full", with
20-30% iowait. The disks are lightly loaded (under 15% util), there are
no D-state tasks, the pressure is confined to the cgroup holding the
QEMU processes, and each QEMU process has one thread in io_cqring_wait.
It started the day I upgraded from Incus 7.5.1 to 7.6; QEMU 11.1.x had
been in use on this host since late September without it.

Host 2: Debian 13, kernel 6.12.116, 56 CPUs, four VMs, Zabbly's Incus
packages. These bundle QEMU 11.1.3 with the pve-qemu patch
"fdmon-io_uring: avoid idle event loop being accounted as IO wait".
Pressure is about 68% "some" and 65% "full" with disks under 1% util,
and again one thread per QEMU process in io_cqring_wait. This kernel
does not advertise IORING_FEAT_NO_IOWAIT (added in 6.15), so the patch
has no effect there.

So the downstream patch only helps on 6.15+ kernels, and distros
shipping unpatched QEMU get these numbers by default.

Is there a preferred way forward? Fiona offered to submit the Proxmox
workaround to qemu-devel, and Stefan asked whether io_uring should
instead account POLL_ADD the way poll(2) does. Either would fix this
for users on distro packages.

Best,
John


^ permalink raw reply	[flat|nested] 17+ messages in thread
* Excessive IO PSI for iothread when using io_uring since QEMU 10.2
@ 2026-04-24 10:25 Fiona Ebner
  2026-04-27 19:13 ` Stefan Hajnoczi
  0 siblings, 1 reply; 17+ messages in thread
From: Fiona Ebner @ 2026-04-24 10:25 UTC (permalink / raw)
  To: open list:Network Block Dev...
  Cc: QEMU Developers, Stefan Hajnoczi, Fam Zheng, Hanna Czenczek,
	Kevin Wolf

Dear maintainers,

since QEMU 10.2, if io_uring is enabled, it will be used for the event
loop of iothreads and this causes an IO pressure stall value of nearly
100 when idle.

The issue was also reported on the kernel mailing list [0]. The
suggestion from Jens Axboe was to just turn off the iowait accounting
completely. But since (for block/file-posix.c), there is actual IO
submitted via the same ring, I wasn't sure if that is the right approach.

So the idea was to keep track of whether the event loop is otherwise
idle and only use the IORING_ENTER_NO_IOWAIT flag in that case [1].

However, doing so would only help for block/file-posix.c, which submits
IO via luring_co_submit() -> fdmon_io_uring_add_sqe(). For example, for
block/rbd.c, only a poll SQE for the AioHandler node's fd is used. When
submitting that poll SQE in the iothread, we would need to be able to
know if IO for RBD is currently in-flight or not to be able to decide
whether to use the IORING_ENTER_NO_IOWAIT flag or not. Is there a good
way to do this (in a general way)?

Or should the flag really always be used (if supported by the kernel)?
Is there a way to tell io_uring/kernel that we are an event loop and our
waiting should only be accounted for when there is actual IO in-flight?

Happy to hear your opinions and suggestions!

[0]:
https://lore.kernel.org/io-uring/14bc6266-5bc9-4454-9518-d1016bfe417b@proxmox.com/T/

[1]:
https://lore.proxmox.com/pve-devel/525c4dad-6d04-41f0-8a21-9302b0c6baa4@proxmox.com/T/

Best Regards,
Fiona



^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2026-11-19 19:00 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-15 23:05 Excessive IO PSI for iothread when using io_uring since QEMU 10.2 John Maguire
2026-11-18 14:06 ` Fernando Casas Schösow
2026-11-18 15:58 ` Jens Axboe
2026-11-18 18:56   ` Stefan Hajnoczi
2026-11-18 19:51     ` Jens Axboe
2026-11-19 12:46       ` Stefan Hajnoczi
2026-11-19 19:00   ` Jens Axboe
  -- strict thread matches above, loose matches on Subject: below --
2026-04-24 10:25 Fiona Ebner
2026-04-27 19:13 ` Stefan Hajnoczi
2026-04-28 12:10   ` Fiona Ebner
2026-04-28 13:31     ` Fiona Ebner
2026-04-28 16:19     ` Stefan Hajnoczi
2026-04-29  8:00       ` Fiona Ebner
2026-04-29 12:20         ` Stefan Hajnoczi
2026-06-01 17:20         ` Stefan Hajnoczi
2026-06-02  8:41           ` Fiona Ebner
2026-06-02 12:08             ` Stefan Hajnoczi

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.