From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:52086) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QPFV9-0005MM-9C for qemu-devel@nongnu.org; Wed, 25 May 2011 10:56:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QPFV8-0002ud-55 for qemu-devel@nongnu.org; Wed, 25 May 2011 10:55:59 -0400 Received: from mail-yx0-f173.google.com ([209.85.213.173]:42181) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QPFV8-0002uR-2f for qemu-devel@nongnu.org; Wed, 25 May 2011 10:55:58 -0400 Received: by yxk8 with SMTP id 8so3682556yxk.4 for ; Wed, 25 May 2011 07:55:57 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20110524210025.GT969@shareable.org> References: <1305233867-4367-1-git-send-email-jvrao@linux.vnet.ibm.com> <20110513085503.GA23158@stefanha-thinkpad.localdomain> <87sjsio0zi.fsf@linux.vnet.ibm.com> <4DCDCA6F.9000503@linux.vnet.ibm.com> <20110524210025.GT969@shareable.org> Date: Wed, 25 May 2011 08:16:02 +0100 Message-ID: From: Stefan Hajnoczi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [0/25] Async threading for VirtFS using glib threads & coroutines. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jamie Lokier Cc: aliguori@us.ibm.com, Venkateswararao Jujjuri , "Aneesh Kumar K.V" , stefanha@linux.vnet.ibm.com, qemu-devel@nongnu.org On Tue, May 24, 2011 at 10:00 PM, Jamie Lokier wrote: > Venkateswararao Jujjuri wrote: >> This model makes the code simple and also in one shot we can convert >> all v9fs_do_syscalls into asynchronous threads. But as Aneesh raised >> will there be any additional overhead for the additional jumps? =A0We >> can quickly test it out too. > > I'm not sure if this is exactly the right place (I haven't followed > the whole discussion), but there is a useful trick for getting rid of > one of the thread context switches: > > Swizzle *which* thread is your "main" coroutine thread. > > Instead of queuing up an item on the work queue, waking the worker > thread pool, and having a worker thread pick up the coroutine, you: > > Declare the current thread to *be* a worker through from this point, > and queue the calling context for a worker thread to pick up. =A0When it > picks it up, *that* thread declares itself to be the main thread > coroutine thread. > > So the coroutine entry step is just queuing a context for another > thread to pick up, and then diving into the blocking system call > (optimising out the enqueue/dequeue and thread switch). > > In a sense, you make the "main" thread a long-lived work queue entry, > and have a symmetric pool, except that the main thread tends to behave > differently than the other work items. > > This only works if the main thread's state is able to follow the > swizzling. =A0I don't know if KVM VCPUs will do that, for example, or if > there's other per-thread state that won't work. > > If the main thread can't be swizzled, you can still use this trick > when doing the coroutine->syscall step starting form an existing > worker thread. The point of this trick is to reduce the latency of issuing the blocking operation? You still have to tell another thread that it has now become the main thread, so this seems like an optimization biased towards I/O latency at the expense of main loop performance. If the "main" thread is a vcpu thread, I'm not sure we want to schedule another thread to become the vcpu thread since this adds an unbounded time until the new vcpu thread is scheduled. The vcpu thread is supposed to be non-blocking so that we don't steal time from the guest and pause execution. Stefan