From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:57328) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S4g3H-0000UM-V0 for qemu-devel@nongnu.org; Mon, 05 Mar 2012 17:06:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S4g2w-0001Z7-DS for qemu-devel@nongnu.org; Mon, 05 Mar 2012 17:06:43 -0500 Received: from smtp02.citrix.com ([66.165.176.63]:60167) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S4g2w-0001WH-8A for qemu-devel@nongnu.org; Mon, 05 Mar 2012 17:06:22 -0500 Message-ID: <1330985175.22559.60.camel@cthulhu.hellion.org.uk> From: Ian Campbell Date: Mon, 5 Mar 2012 14:06:15 -0800 In-Reply-To: <4F4CBE85.5000006@citrix.com> References: <4F4CBE85.5000006@citrix.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit MIME-Version: 1.0 Subject: Re: [Qemu-devel] [Xen-devel] Qemu disaggregation in Xen environment List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Julien Grall Cc: "xen-devel@lists.xensource.com" , "qemu-devel@nongnu.org" , Stefano Stabellini , Julian Pidancet On Tue, 2012-02-28 at 06:46 -0500, Julien Grall wrote: > Hello, > > In the current model, only one instance of qemu is running for each running HVM domain. > > We are looking at disaggregating qemu to have, for example, an instance to emulate only > network controllers, another to emulate block devices, etc... > > Multiple instances of qemu would run for a single Xen domain. Each one would handle > a subset of the hardware. > > Has someone already looked at it and potentially already submitted code for qemu ? I'm not aware of any code existing to do this. There's a bunch of interesting stuff to do on the Xen side to make this stuff work. Firstly you would need to add support to the hypervisor for dispatching I/O requests to multiple qemu instances (via multiple io req rings). I think at the moment there is only support for a single ring (or maybe it's one sync and one buffered I/O ring). You'd also need to make sure that qemu explicitly requests all the MMIO regions it is interested in. Currently the hypervisor forwards any unknown MMIO to qemu so the explicit registration is probably not done as consistently as it could be. If you want to have N qemus then you need to make sure that at least N-1 of register for everything they are interested in. Currently the PCI config space decode is done within qemu which is a bit tricky if you are wanting to have different emulated PCI devices in different qemu processes. We think it would independently be an architectural improvement to have the hypervisor do the PCI config space decode anyway. This would allow it to forward the I/O to the correct qemu (there are other benefits to this change, e.g. relating to PCI passthrough and the handling of MSI configuration etc) Then you'd need to do a bunch of toolstack level work to start and manage the multiple qemu processes instead of the existing single process. So, a bunch of stuff but I think it is all reasonable to do individually and each brings its own advantages to the architecture outside of this project too. > The purpose of this e-mail is to start a discussion and gather opinions on how the > qemu developers community would like to see it implemented. > > A couple of questions comes to mind: I guess these are mostly qemu side questions. I'm not familiar enough with the internals on that side to answer really. > 1) How hard would it be to untangle "machine" specific (PC hardware) emulation > from "device" specific emulation (PCI devices) ? > > 2) How can we achieve disaggregation from a configuration point of view. Currently, > Xen toolstack starts qemu, and tells qemu which device to emulate using the command > line. I've heard about a project for creating machine description configuration files > for QEMU which could help greatly in dividing up which hardware to emulate in which > instance of qemu. I've only vaguely heard about this but it certainly seems like functionality which would benefit more than just Xen. Ian. > What is the status of this project ? > > Thank you for your answers, > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel