Re: Async machine appliance for testing (python)

["Daniel P. Berrangé" <berrange@redhat.com>]

On Mon, Aug 31, 2026 at 04:51:36PM -0400, John Snow wrote:
> WIth the upgrade to 3.12 imminent, now might be a good time for me to
> revisit the work I started when I converted QMP from sync to an async
> library. I have a draft of the library I wrote some time ago, but it's
> probably in somewhat dodgy shape after a year or two of being ignored.
> 
> Before I dig in, I wanted to ask if this would be a helpful project.
> The main selling points would be:
> 
> (1) Ability to split it out from the main QEMU tree, like QMP was, for
> use in external testing
> (2) Better stream management so console information is consumed and
> logged reliably without deadlocks due to pipes hitting their buffer
> limit

I can't remember the specific details, but I do recall some
scenarios where it would have been valuable to have the console
output continuously streamed to the log file on disk, while
at the same time retainining the ability to do expected
output matching from the API.

> (3) Removing multiprocess and multithread processing of console and
> stdout/stderr logging from test suites in favor of using async
> user-threads for (hopefully) simpler, more straightforward processing
> and customization.

Which tests were you refering to here ?  I didn't think we had
python tests using multiprocess/multithread ?

Tangentially related, in debugging failures of the functional
test suite, and to a lesser extent I/O tests / qmp-shell, I've
often found myself heavily confused by the asyncio call paths
and control flow, making it really hard to debug.

In particular I've never fully figured out how to get qmp-shell
to have a clean exit path without stack traces being thrown.

I'm unclear if this is related to our having QEMUMachine class
expose a sync API on top of an async QMP class (square peg in
a round hole situation), or if this is just something inherent
to Python asyncio APIs, or probably a bit of both ?

If switching to a fully async API throughout the stack makes
debugging simpler, I'd welcome it.

> (4) Ability to add listeners/event handlers to watch for specific
> patterns and trigger asynchronously when they occur; i.e. ability to
> handle and watch for specific patterns in an arbitrary order without
> deadlocking your test
> (5) Centralized timeout setting and handling.

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|