From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1LCeSl-0004ga-CR for qemu-devel@nongnu.org; Tue, 16 Dec 2008 13:16:07 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1LCeSk-0004dy-8i for qemu-devel@nongnu.org; Tue, 16 Dec 2008 13:16:06 -0500 Received: from [199.232.76.173] (port=35008 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LCeSk-0004dn-4G for qemu-devel@nongnu.org; Tue, 16 Dec 2008 13:16:06 -0500 Received: from mail-bw0-f12.google.com ([209.85.218.12]:34431) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1LCeSj-0006QI-Nq for qemu-devel@nongnu.org; Tue, 16 Dec 2008 13:16:06 -0500 Received: by bwz5 with SMTP id 5so6085940bwz.10 for ; Tue, 16 Dec 2008 10:15:50 -0800 (PST) Message-ID: Date: Tue, 16 Dec 2008 20:11:58 +0200 From: "Blue Swirl" Subject: Re: [Qemu-devel] Re: [PATCH 2 of 5] add can_dma/post_dma for direct IO In-Reply-To: <4947E9D1.8060007@codemonkey.ws> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <49455A33.207@codemonkey.ws> <49456337.4000000@redhat.com> <494591F7.3080002@codemonkey.ws> <4946D501.4020109@codemonkey.ws> <494777E7.8050305@redhat.com> <4947E0D1.6060704@redhat.com> <4947E9D1.8060007@codemonkey.ws> Reply-To: qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Anthony Liguori Cc: Andrea Arcangeli , chrisw@redhat.com, kvm@vger.kernel.org, qemu-devel@nongnu.org, Gerd Hoffmann , Avi Kivity On 12/16/08, Anthony Liguori wrote: > Avi Kivity wrote: > > > Blue Swirl wrote: > > > > > > > > > I don't understand. It's not a device that needs bouncing, it's a > > > > particular transfer. This could be either due to the transfer > targeting > > > > mmio, or due to the transfer requiring a transformation. > > > > > > > > > > > > > > Should the bouncing be something more much complex, for example > > > negotiated between the devices? Or maybe the devices set up a > > > transforming and non-transforming channel (which the other end should > > > be able to transform some more) and use them as needed? > > > > > > > > > > Yes. We already have two cases: > > > > - may do partial request: useful for block storage where requests can be > huge > > - need full request: networking, where you can't send or receive half a > packet; on the other hand, packets are small (even with tso) > > > > You're adding a third case, always bounce, when the data undergoes a > transformation which can't happen in-place. > > > > IMHO, IOMMU translation is distinct from mapping/data transformation. I > would think the IOMMU translation API would be different in that it took a > physical address and returned a physical address. Fully agree. On Sparc32, the incoming address is called DVMA (device virtual memory address). > The IOMMU DMA API (which could transform data potentially) should return a > virtual address and data a physical address. In the normal case > (non-transforming IOMMU), it should just fall-through to CPU DMA API (which > is just map/unmap). > > The PCI DMA API would normally fall through to the IOMMU DMA API. > > So we would have: > > PCI DMA map(addr): > if IOMMU: > return IOMMU DMA map(addr) > return CPU DMA map(addr) > > IOMMU DMA map(addr): > new_address = IOMMU translate(addr) > if transform: > return IOMMU byte-swap map(addr) > return CPU DMA map(addr) But with this we would be back to the original merged API. I'd propose three to four distinct steps: 1. Resolve the device addresses via IOMMU etc. to guest physical addresses (typically no changes if no IOMMU) 2. Negotiate bouncing (if bounced, the step returns a host address, if not, a physical address) 3. Map the physical addresses to host addresses, bypass for the bounce cases 4. Perform vectorized zero-copy AIO. (Profit?) 2 & 3 may/should be merged but I'm not sure we have the full picture yet.