[Qemu-devel] [PATCH 0/2] aio: add aio_context_acquire() and aio_context_release()

[Stefan Hajnoczi <stefanha@redhat.com>]

aio_context_acquire() and aio_context_release() make it possible for multiple
threads to safely operate on a shared AioContext.  This is a prerequisite for
using the block layer outside the QEMU global mutex.

Imagine that a dataplane thread is performing I/O on behalf of the guest when
the user issues a monitor command that needs to access the BlockDriverState.
The monitor thread needs to acquire the AioContext before calling bdrv_*()
functions on it.  This prevents the dataplane thread from interfering with the
monitor thread.

There was a discussion in the RFC email thread about how to implement fairness:
each thread should get a turn to acquire the AioContext so that starvation is
avoided.

Paolo suggested doing what the QEMU main loop does today: drop the lock before
invoking ppoll(2).  It turns out that this is tricky since we want both threads
to be able to call aio_poll() simultaneously.  AioContext->pollfds[] currently
prevents two simultaneous aio_poll() calls since it is a shared resource.  It's
also tricky to avoid deadlocks if two threads execute aio_poll() simulatenously
except by waking all threads each time *any* thread makes any progress.

I found the simplest solution is to implement RFifoLock, a recursive lock with
FIFO ordering.  This lock supports the semantics we need for the following
pattern:

  /* Event loop thread */
  while (running) {
      aio_context_acquire(ctx);
      aio_poll(ctx, true);
      aio_context_release(ctx);
  }

  /* Another thread */
  aio_context_acquire(ctx);
  bdrv_read(bs, 0x1000, buf, 1);
  aio_context_release(ctx);

When the other thread wants to acquire the AioContext but the lock is
contended, it uses aio_notify(ctx) to bump the event loop thread out of
aio_poll().  Fairness ensures everyone gets an opportunity to use the
AioContext.

Stefan Hajnoczi (2):
  rfifolock: add recursive FIFO lock
  aio: add aio_context_acquire() and aio_context_release()

 async.c                  | 18 ++++++++++
 include/block/aio.h      | 18 ++++++++++
 include/qemu/rfifolock.h | 54 +++++++++++++++++++++++++++++
 tests/Makefile           |  2 ++
 tests/test-aio.c         | 58 +++++++++++++++++++++++++++++++
 tests/test-rfifolock.c   | 90 ++++++++++++++++++++++++++++++++++++++++++++++++
 util/Makefile.objs       |  1 +
 util/rfifolock.c         | 79 ++++++++++++++++++++++++++++++++++++++++++
 8 files changed, 320 insertions(+)
 create mode 100644 include/qemu/rfifolock.h
 create mode 100644 tests/test-rfifolock.c
 create mode 100644 util/rfifolock.c

-- 
1.8.3.1