From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:50257) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TkGl7-0007LS-Vj for qemu-devel@nongnu.org; Sun, 16 Dec 2012 11:08:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TkGl5-0005E8-41 for qemu-devel@nongnu.org; Sun, 16 Dec 2012 11:08:09 -0500 Received: from mx1.redhat.com ([209.132.183.28]:30948) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TkGl4-0005E3-TI for qemu-devel@nongnu.org; Sun, 16 Dec 2012 11:08:07 -0500 Date: Sun, 16 Dec 2012 18:11:14 +0200 From: "Michael S. Tsirkin" Message-ID: <20121216161114.GD15790@redhat.com> References: <1355144985-12897-1-git-send-email-stefanha@redhat.com> <1355144985-12897-4-git-send-email-stefanha@redhat.com> <20121211141340.GA18753@redhat.com> <20121211154258.GA20251@redhat.com> <87d2ygpn1f.fsf@codemonkey.ws> <20121211180956.GB9257@redhat.com> <20121212153421.GA16956@stefanha-thinkpad.redhat.com> <20121212154906.GB17446@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [PATCH v6 03/12] dataplane: add host memory mapping code List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: Kevin Wolf , Anthony Liguori , qemu-devel , Blue Swirl , Khoa Huynh , Stefan Hajnoczi , Paolo Bonzini , Asias He On Fri, Dec 14, 2012 at 12:45:16PM +0100, Stefan Hajnoczi wrote: > On Wed, Dec 12, 2012 at 4:49 PM, Michael S. Tsirkin wrote: > > On Wed, Dec 12, 2012 at 04:34:21PM +0100, Stefan Hajnoczi wrote: > >> On Tue, Dec 11, 2012 at 08:09:56PM +0200, Michael S. Tsirkin wrote: > >> > On Tue, Dec 11, 2012 at 10:32:28AM -0600, Anthony Liguori wrote: > >> > > "Michael S. Tsirkin" writes: > >> > > > >> > > > On Tue, Dec 11, 2012 at 04:27:49PM +0100, Stefan Hajnoczi wrote: > >> > > >> On Tue, Dec 11, 2012 at 3:13 PM, Michael S. Tsirkin wrote: > >> > > >> > On Mon, Dec 10, 2012 at 02:09:36PM +0100, Stefan Hajnoczi wrote: > >> > > >> >> The data plane thread needs to map guest physical addresses to host > >> > > >> >> pointers. Normally this is done with cpu_physical_memory_map() but the > >> > > >> >> function assumes the global mutex is held. The data plane thread does > >> > > >> >> not touch the global mutex and therefore needs a thread-safe memory > >> > > >> >> mapping mechanism. > >> > > >> >> > >> > > >> >> Hostmem registers a MemoryListener similar to how vhost collects and > >> > > >> >> pushes memory region information into the kernel. There is a > >> > > >> >> fine-grained lock on the regions list which is held during lookup and > >> > > >> >> when installing a new regions list. > >> > > >> > > >> > > >> > Can we export and reuse the vhost code for this? > >> > > >> > I think you will find this advantageous when you add migration > >> > > >> > support down the line. > >> > > >> > And if you find it necessary to use MemoryListener e.g. for performance > >> > > >> > reasons, then vhost will likely benefit too. > >> > > >> > >> > > >> It's technically possible and not hard to do but it prevents > >> > > >> integrating deeper with core QEMU as the memory API becomes > >> > > >> thread-safe. > >> > > >> > >> > > >> There are two ways to implement dirty logging: > >> > > >> 1. The vhost log approach which syncs dirty information periodically. > >> > > >> 2. A cheap thread-safe way to mark dirty outside the global mutex, > >> > > >> i.e. a thread-safe memory_region_set_dirty(). > >> > > > > >> > > > You don't normally want to dirty the whole region, > >> > > > you want to do this to individual pages. > >> > > > > >> > > >> If we can get thread-safe guest memory load/store in QEMU then #2 is > >> > > >> included. We can switch to using hw/virtio.c instead of > >> > > >> hw/dataplane/vring.c, we get dirty logging for free, we can drop > >> > > >> hostmem.c completely, etc. > >> > > >> > >> > > >> Stefan > >> > > > > >> > > > So why not reuse existing code? If you drop it later it won't > >> > > > matter what you used ... > >> > > > >> > > Let's not lose sight of the forest for the trees here... > >> > > > >> > > This whole series is not reusing existing code. That's really the whole > >> > > point. > >> > > > >> > > The point is to take the code (duplication and all) and then do all of > >> > > the refactoring to use common code in the tree itself. > >> > > > >> > > If we want to put this in a hw/staging/ directory, that's fine by me > >> > > too. > >> > > > >> > > Regards, > >> > > > >> > > Anthony Liguori > >> > > >> > Yes I agree. I think lack of handling for cross regin descriptors > >> > bothers me a bit more. > >> > >> The two things you've mentioned both aren't handled by hw/virtio.c: > >> > >> 1. Issue: Indirect descriptors have no alignment restrictions and can > >> cross regions. > >> > >> hw/virtio.c uses vring_desc_flags() and other accessor functions, > >> which do lduw_phys() - there is no memory region boundary checking > >> here. > > > > Since addresses are aligned this one is fine I think. > > > >> 2. Issue: Virtio buffers can cross memory region boundaries. > >> > >> hw/virtio.c maps buffers 1:1 using virtqueue_map_sg() and exits if > >> mapping fails. It does not split buffers if they cross a memory > >> region. > >> > >> These are definitely ugly corner cases but hw/virtio.c is proof that > >> we're not hitting them in practice. > >> > >> Stefan > > > > Yes, this one seems ugly. Maybe add a TODO? > > > > OK let's assume we want to put it in staging/ > > I worry about the virtio-blk changes being isolated. > > Can you put ifdef CONFIG_VIRTIO_BLK_DATA_PLANE around > > them all to avoid dependency on that header completely > > if configured out? > > Okay, I'll move the #ifdefs. I like the stubs in the header file > because it reduces the amount of #ifdefs, but this is easy to change. > > Stefan Okay. Another option (if you prefer stubs) is to add a stub for access to s->dataplane field, and surround just the field with ifdefs. As it is, this code: if (s->dataplane) { return; } can't be compiled out since compiler is not smart enough to figure out dataplane is never set. -- MST