From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:47029) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TwEaS-0001Dp-Pt for qemu-devel@nongnu.org; Fri, 18 Jan 2013 11:15:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TwEZz-0001D1-IO for qemu-devel@nongnu.org; Fri, 18 Jan 2013 11:14:36 -0500 Received: from mail-qc0-f176.google.com ([209.85.216.176]:52116) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TwEZz-0001Cx-Dz for qemu-devel@nongnu.org; Fri, 18 Jan 2013 11:14:07 -0500 Received: by mail-qc0-f176.google.com with SMTP id n41so2474283qco.21 for ; Fri, 18 Jan 2013 08:14:07 -0800 (PST) Sender: Paolo Bonzini Message-ID: <50F974CA.8010603@redhat.com> Date: Fri, 18 Jan 2013 17:14:02 +0100 From: Paolo Bonzini MIME-Version: 1.0 References: <20130118160448.GA76938@onelab2.iet.unipi.it> In-Reply-To: <20130118160448.GA76938@onelab2.iet.unipi.it> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] bypassing pci_dma_read() and pci_dma_write() ? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Luigi Rizzo Cc: Giuseppe Lettieri , qemu-devel@nongnu.org, v.maffione@gmail.com Il 18/01/2013 17:04, Luigi Rizzo ha scritto: > Hi, > with a bunch of e1000 improvements we are at a point where we are > doing over 1Mpps (short frames) and 7-8Gbit/s (1500 byte frames) > between two guests, and two things that are high in the "perf top" > stats are phys_page_find() and related memory copies. > > Both are triggered by the pci_dma_read() and pci_dma_write(), > which on e1000 (and presumably other frontends) are called on > every single descriptor and every single buffer. > > I have then tried to access the guest memory without going every > time through the page lookup. [...] > > This relies on the assumption that the ring (which is contiguous in the > guest's physical address space) is also contiguous in the host's virtual > address space. In principle the property could be easily verified once > the ring is set up. IIRC, the amount of contiguous memory is written by address_space_map in the plen parameter. In your case: > + s->txring = address_space_map(pci_dma_context(&s->dev)->as, > + base, &desclen, 0 /* is_write */); that would be desclen on return from address_space_map. > And of course, am i missing some important detail ? Unfortunately yes. First, host memory mappings could change (though they rarely do on PC). The result of address_space_map is not guaranteed to be stable. To avoid problems with this, however, you could use something like hw/dataplane/hostmem.c and even avoid address_space_map altogether. Second, that pci_dma_*() could have the addresses translated by an IOMMU. virtio is documented to have "real" physical memory addresses, but this does not apply to other devices. Paolo > Of course the above could be used conditionally if the required > conditions hold, and then revert to the pci_dma_*() > in other cases. > > cheers > luigi > >