Best practice for issuing blocking calls in response to an event

[Miles Glenn <milesg@linux.ibm.com>]

Hello,

I am attempting to simulate a system with multiple CPU
architectures.  To do this I am starting a unique QEMU process for each
CPU architecture that is needed. I'm also developing some QEMU code
that aids in transporting MMIO transactions across the process
boundaries using sockets.

The design takes MMIO request messages off of a socket, services the
request by calling address_space_ldq_be(), then sends a response
message (containing the requested data) over the same
socket.  Currently, this is all done inside the socket IOReadHandler
callback function.

This works as long as the targeted register exists in the same QEMU
process that received the request.  However, If the register exists in
another QEMU process, then the call to address_space_ldq_be() results
in another socket message being sent to that QEMU process, requesting
the data, and then waiting (blocking) for the response message
containing the data.  In other words, it ends up blocking inside the
event handler and even though the QEMU process containing the target
register was able to receive the request and send the response, the
originator of the request is unable to receive the response until it
eventually times out and stops blocking.  Once it times out and stops
blocking, it does receive the response, but now it is too late.

Here's a summary of the stack up to where the code blocks:

IOReadHandler callback
  calls address_space_ldq_be()
    resolves to mmio read op of a remote device
      sends request over socket and waits (blocks) for response

So, I'm looking for a way to handle the work of calling
address_space_ldq_be(), which might block when attempting to read a
register of a remote device, without blocking inside the IOReadHandler
callback context.

I've done a lot of searches and reading about how to do this on the web
and in the QEMU code but it's still not really clear to me how this
should be done in QEMU.  I've seen a lot about using coroutines to
handle cases like this. Is that what I should be using here?

Thanks,

Glenn Miles