From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51721) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VTqUc-0001BX-M1 for qemu-devel@nongnu.org; Wed, 09 Oct 2013 05:55:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VTqUW-0005jl-Bv for qemu-devel@nongnu.org; Wed, 09 Oct 2013 05:55:46 -0400 Received: from mx1.redhat.com ([209.132.183.28]:23838) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VTqUV-0005ja-RO for qemu-devel@nongnu.org; Wed, 09 Oct 2013 05:55:40 -0400 From: Stefan Hajnoczi Date: Wed, 9 Oct 2013 11:55:29 +0200 Message-Id: <1381312531-28723-1-git-send-email-stefanha@redhat.com> Subject: [Qemu-devel] [PATCH 0/2] aio: add aio_context_acquire() and aio_context_release() List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Kevin Wolf , Paolo Bonzini , Michael Roth , Stefan Hajnoczi , xiawenc@linux.vnet.ibm.com aio_context_acquire() and aio_context_release() make it possible for multiple threads to safely operate on a shared AioContext. This is a prerequisite for using the block layer outside the QEMU global mutex. Imagine that a dataplane thread is performing I/O on behalf of the guest when the user issues a monitor command that needs to access the BlockDriverState. The monitor thread needs to acquire the AioContext before calling bdrv_*() functions on it. This prevents the dataplane thread from interfering with the monitor thread. There was a discussion in the RFC email thread about how to implement fairness: each thread should get a turn to acquire the AioContext so that starvation is avoided. Paolo suggested doing what the QEMU main loop does today: drop the lock before invoking ppoll(2). It turns out that this is tricky since we want both threads to be able to call aio_poll() simultaneously. AioContext->pollfds[] currently prevents two simultaneous aio_poll() calls since it is a shared resource. It's also tricky to avoid deadlocks if two threads execute aio_poll() simulatenously except by waking all threads each time *any* thread makes any progress. I found the simplest solution is to implement RFifoLock, a recursive lock with FIFO ordering. This lock supports the semantics we need for the following pattern: /* Event loop thread */ while (running) { aio_context_acquire(ctx); aio_poll(ctx, true); aio_context_release(ctx); } /* Another thread */ aio_context_acquire(ctx); bdrv_read(bs, 0x1000, buf, 1); aio_context_release(ctx); When the other thread wants to acquire the AioContext but the lock is contended, it uses aio_notify(ctx) to bump the event loop thread out of aio_poll(). Fairness ensures everyone gets an opportunity to use the AioContext. Stefan Hajnoczi (2): rfifolock: add recursive FIFO lock aio: add aio_context_acquire() and aio_context_release() async.c | 18 ++++++++++ include/block/aio.h | 18 ++++++++++ include/qemu/rfifolock.h | 54 +++++++++++++++++++++++++++++ tests/Makefile | 2 ++ tests/test-aio.c | 58 +++++++++++++++++++++++++++++++ tests/test-rfifolock.c | 90 ++++++++++++++++++++++++++++++++++++++++++++++++ util/Makefile.objs | 1 + util/rfifolock.c | 79 ++++++++++++++++++++++++++++++++++++++++++ 8 files changed, 320 insertions(+) create mode 100644 include/qemu/rfifolock.h create mode 100644 tests/test-rfifolock.c create mode 100644 util/rfifolock.c -- 1.8.3.1