All of lore.kernel.org
 help / color / mirror / Atom feed
* [RESEND PATCH V10 00/15] iothread: Support tracking and querying IOThread holders
@ 2026-07-16 17:20 Zhang Chen
  2026-07-16 17:20 ` [RESEND PATCH V10 01/15] qapi/misc: Fix missed query-iothreads items Zhang Chen
                   ` (15 more replies)
  0 siblings, 16 replies; 20+ messages in thread
From: Zhang Chen @ 2026-07-16 17:20 UTC (permalink / raw)
  To: qemu-devel, Dr . David Alan Gilbert, Eric Blake,
	Markus Armbruster, Michael S . Tsirkin, Stefan Hajnoczi,
	Paolo Bonzini, Kevin Wolf, Jason Wang, Fam Zheng,
	Daniel P . Berrangé
  Cc: Zhang Chen

### Motivation
Currently, the relationship between IOThreads and the resources using them is
opaque. From a management perspective, it is difficult to determine which
devices or block exports are currently associated with a specific IOThread
via QMP or HMP.

This series introduces a "holder" tracking mechanism based on QOM paths or
block exports to significantly improve IOThread observability.
In high-performance virtualization environments, users often pre-allocate
a set of IOThreads to serve as a persistent thread pool. During device
hotplug/unplug cycles, while IOThread objects remain persistent,
their attachment state changes dynamically.

By tracking these "holders," administrators can use `query-iothreads` to  
monitor the real-time load distribution of the thread pool and identify which 
specific devices (e.g., virtio-blk, BlockExport) are attached to which thread.

### Technical Overview
* Atomic API Design: Merged iothread_get_aio_context and 
  iothread_ref_and_get_aio_context into a single unified function. The new 
  API enforces a 'holder' parameter, making "retrieval" and "tracking" an  
  atomic operation to prevent reference tracking leaks.
* Subsystem Integration: Fully migrated and adapted core modules including 
  Virtio (blk, vq-mapping, balloon, scsi), Block Layer (export, xen), 
  Network (COLO), VFIO, and the Monitor subsystem.
* Introspection: Expanded the IOThreadInfo QAPI structure to include a 
  'holders' array (list of QOM paths or block exports), accessible via both 
  query-iothreads (QMP) and info iothreads (HMP).

### V10 -> V9 Changelog
  - Rebase this series on the Daniel P. Berrangé's series:
    [PATCH v7 00/35] monitor: turn QMP and HMP into QOM objects
    https://lore.kernel.org/qemu-devel/20260706135824.2623960-1-berrange@redhat.com/
  - Remove the monitor type of IOThreadHolder, make the monitor as a QOM object.
  - Remove GList related code and stays within QAPI instead of mixing GList and QAPI.
  - Fix missed comments in the patch 2.
  - Rewrite parts of commit log generated by AI. 
  - Keep Stefan's reviewed-by in patch 2 and addressed Markus's comments.
  - Rewrite parts of code comments.
  - Drop original patch 6/7/8 for monitor parts.

### V9 -> V8 Changelog
  - Drop to assign unique default ID for monitor, and keep the NULL ID. 
  - Rewrite and split the monitor patch 7 and 8.
  - Add the [PATCH 17/17]tests/unit/iothread: Update the iothread_get_aio_context
    to keep the API same in the unit tests.
  - Refactorcode according comments.
  - Update the example.
  - Fix other comments issues.

### V7 -> V8 Changelog
  - Drop the assumption of the Monitor QOM, add the new type
    IO_THREAD_HOLDER_KIND_MONITOR_NAME.
  - Drop the assumption of the string '/' as QOM path.
  - Introduce new patch for Monitor:
    [PATCH 06/15] assign unique default ID to anonymous monitors
  - Fixed Monitor ID for the IOThread.
  - Refactorcode according comments.
  - Update QMP example.
  - Rebased patches on upstream code.
  - Fixed comments issues.
  - Removed redundant code.

### V6 -> V7 Changelog
  - Fixed comments issues in patch 02/14 (Thanks Markus).
  - Removed redundant code.
  - Rebased patches on upstream code.
  - Fixed code conflict with latest patches.
  - Added detailed description in git log and code.
  - Added comments for patch 06/14 assuming the monitor is a QOM object,
    Depends on Daniel Berrange's RFC patch:
    [PATCH RFC 00/17] monitor: turn QMP and HMP into QOM objects

### V5 -> V6 Changelog
* API Refactoring (Breaking Change):
  - Implement the 'union': 'IoThreadHolder' for combine the QOM objects
    and the block-node.
  - Completely merged iothread_get_aio_context variants.
  - The iothread_get_aio_context function now strictly requires a 'holder'
    argument to ensure every context retrieval is accounted for.
* Expanded Module Coverage:
  - Added holder tracking for the monitor subsystem (tracking threads used
    by QMP/HMP).
  - Added support for the virtio-scsi dataplane.
* Code Cleanup:
  - Removed redundant and obsolete API declarations.
  - Standardized the use of object_get_canonical_path() across all callers
    to provide consistent and unique holder names.
* Bug Fixes:
  - Updated qapi/misc.json documentation to fix missing descriptions in
    query-iothreads items.
* Build System:
  - Improved header inclusion logic to avoid redundant declaration errors
    in unit tests.
* QAPI part:
  - Sorry to drop the "Acked-by: Markus Armbruster <armbru@redhat.com>",
    because this version have lots of changes in the:
    [PATCH V6 13/14]qapi: examine IOThread attachment status via query-iothreads



Zhang Chen (15):
  qapi/misc: Fix missed query-iothreads items
  iothread: introduce iothread_ref/unref to track attached devices
  iothread: tracking iothread users with holder name
  iothread: introduce iothread_unsafe_get_aio_context()
  block/export: track IOThread reference in BlockExport
  monitor: Update tracking iothread users with holder
  virtio-vq-mapping: track iothread-vq-mapping references using device
    path
  virtio: use iothread_get/put_aio_context for thread pinning
  net/colo: track IOThread references using path-based holder
  virtio-balloon: Update tracking iothread users with holder
  vfio-user/proxy: Update tracking iothread users with holder name
  xen-block: Update tracking iothread users with holder name
  monitor/hmp: Add holders support for hmp_info_iothreads
  iothread: simplify API by merging iothread_get_aio_context variants
  tests/unit/iothread: Update the iothread_get_aio_context

 block/export/export.c                   | 78 +++++++++++++++++---
 blockdev.c                              |  2 +-
 hw/block/dataplane/xen-block.c          | 19 ++++-
 hw/block/virtio-blk.c                   | 22 ++++--
 hw/scsi/virtio-scsi-dataplane.c         | 22 ++++--
 hw/vfio-user/proxy.c                    | 15 +++-
 hw/virtio/iothread-vq-mapping.c         | 19 +++--
 hw/virtio/virtio-balloon.c              | 22 ++++--
 include/block/export.h                  |  6 ++
 include/hw/virtio/iothread-vq-mapping.h |  6 +-
 include/system/iothread.h               | 30 +++++++-
 iothread.c                              | 96 ++++++++++++++++++++++++-
 monitor/hmp-cmds.c                      | 22 ++++++
 monitor/monitor-internal.h              |  3 +
 monitor/monitor.c                       | 26 +++++--
 monitor/qmp.c                           |  5 +-
 net/colo-compare.c                      | 29 +++++---
 qapi/misc.json                          | 71 +++++++++++++++++-
 tests/unit/iothread.c                   | 16 +++--
 tests/unit/iothread.h                   |  6 +-
 tests/unit/test-aio-multithread.c       |  5 +-
 tests/unit/test-bdrv-drain.c            | 18 +++--
 tests/unit/test-block-iothread.c        | 21 ++++--
 23 files changed, 478 insertions(+), 81 deletions(-)

-- 
2.49.0



^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2026-07-21 12:52 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-16 17:20 [RESEND PATCH V10 00/15] iothread: Support tracking and querying IOThread holders Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 01/15] qapi/misc: Fix missed query-iothreads items Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 02/15] iothread: introduce iothread_ref/unref to track attached devices Zhang Chen
2026-07-21 12:20   ` Markus Armbruster
2026-07-16 17:20 ` [RESEND PATCH V10 03/15] iothread: tracking iothread users with holder name Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 04/15] iothread: introduce iothread_unsafe_get_aio_context() Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 05/15] block/export: track IOThread reference in BlockExport Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 06/15] monitor: Update tracking iothread users with holder Zhang Chen
2026-07-21 12:33   ` Markus Armbruster
2026-07-16 17:20 ` [RESEND PATCH V10 07/15] virtio-vq-mapping: track iothread-vq-mapping references using device path Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 08/15] virtio: use iothread_get/put_aio_context for thread pinning Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 09/15] net/colo: track IOThread references using path-based holder Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 10/15] virtio-balloon: Update tracking iothread users with holder Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 11/15] vfio-user/proxy: Update tracking iothread users with holder name Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 12/15] xen-block: " Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 13/15] monitor/hmp: Add holders support for hmp_info_iothreads Zhang Chen
2026-07-21 12:51   ` Markus Armbruster
2026-07-16 17:20 ` [RESEND PATCH V10 14/15] iothread: simplify API by merging iothread_get_aio_context variants Zhang Chen
2026-07-16 17:20 ` [RESEND PATCH V10 15/15] tests/unit/iothread: Update the iothread_get_aio_context Zhang Chen
2026-07-21 12:47 ` [RESEND PATCH V10 00/15] iothread: Support tracking and querying IOThread holders Markus Armbruster

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.