From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55879) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WiTWS-00060w-6w for qemu-devel@nongnu.org; Thu, 08 May 2014 14:58:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WiTWM-0001O3-1l for qemu-devel@nongnu.org; Thu, 08 May 2014 14:58:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:5099) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WiTWL-0001Nz-P0 for qemu-devel@nongnu.org; Thu, 08 May 2014 14:58:17 -0400 Date: Thu, 8 May 2014 19:58:11 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20140508185811.GF2408@work-vm> References: <20140508101655.GA7534@stefanha-thinkpad.redhat.com> <20140508134435.GB2408@work-vm> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [RFC] dataplane: IOThreads and writing dataplane-capable code List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: Kevin Wolf , Fam Zheng , qemu-devel , Max Reitz , Stefan Hajnoczi , Paolo Bonzini * Stefan Hajnoczi (stefanha@gmail.com) wrote: > On Thu, May 8, 2014 at 3:44 PM, Dr. David Alan Gilbert > wrote: > > * Stefan Hajnoczi (stefanha@redhat.com) wrote: > > > > > > > >> How to synchronize with an IOThread > >> ----------------------------------- > >> AioContext is not thread-safe so some rules must be followed when using file > >> descriptors, event notifiers, timers, or BHs across threads: > >> > >> 1. AioContext functions can be called safely from file descriptor, event > >> notifier, timer, or BH callbacks invoked by the AioContext. No locking is > >> necessary. > >> > >> 2. Other threads wishing to access the AioContext must use > >> aio_context_acquire()/aio_context_release() for mutual exclusion. Once the > >> context is acquired no other thread can access it or run event loop iterations > >> in this AioContext. > >> > >> aio_context_acquire()/aio_context_release() calls may be nested. This > >> means you can call them if you're not sure whether #1 applies. > >> > >> Side note: the best way to schedule a function call across threads is to create > >> a BH in the target AioContext beforehand and then call qemu_bh_schedule(). No > >> acquire/release or locking is needed for the qemu_bh_schedule() call. But be > >> sure to acquire the AioContext for aio_bh_new() if necessary. > > > > How do these IOThreads pause during migration? > > Are they paused by the 'qemu_mutex_lock_iothread' that the migration thread calls? > > Currently the only IOThread user is virtio-blk data-plane. It has a > VM state change listener registered that will stop using the IOThread > during migration. > > In the future we'll have to do more than that: > It is possible to suspend all IOThreads simply by looping over > IOThread objects and calling aio_context_acquire() on their > AioContext. You can release the AioContexts when you are done. This > would be suitable for a "stop the world" operation for migration > hand-over. That worries me for two reasons: 1) I'm assuming there is some subtlety so that it doesn't deadlock when another thread is trying to get a couple of contexts. 2) The migration code that has to pause everything is reasonably time critical (OK not super critical - but it worries if it gains more than a few ms). Doing something to each thread in series where that thread might have to finish up a transaction sounds like it could add together to be quite large. > For smaller one-off operations like block-migration.c it may also make > sense to acquire/release the AioContext. But that's not necessary > today since dataplane is disabled during migration. I guess it's probably right to hide this behind some interface on the Aio stuff that migration can call and it can worry about speed, and locking order etc. I also would we end up wanting some IOThreads to continue - e.g. could we be using them for transport of the migration stream or are they strictly for the guests use? Dave -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK