From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: Is there a way to map pv guest pseudo physical address into dom0? Date: Mon, 27 Jun 2011 12:50:08 -0400 Message-ID: <20110627165008.GA13335@dumpdata.com> References: <1309165317.32717.160.camel@zakaz.uk.xensource.com> <1309166253.32717.167.camel@zakaz.uk.xensource.com> <20110627093345.GH17634@whitby.uk.xensource.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Wei Liu Cc: Tim Deegan , Ian Campbell , "xen-devel@lists.xensource.com" , Anthony Perard , Stefano Stabellini List-Id: xen-devel@lists.xenproject.org On Mon, Jun 27, 2011 at 09:35:24PM +0800, Wei Liu wrote: > Thank you, Tim and Stefano. >=20 > On Mon, Jun 27, 2011 at 7:23 PM, Stefano Stabellini > wrote: > > On Mon, 27 Jun 2011, Tim Deegan wrote: > >> At 17:26 +0800 on 27 Jun (1309195596), Wei Liu wrote: > >> > > You need to arrange for the Xen specific bit of the virtio front= end to > >> > > do the p2m translation. All externally visible addresses from a = PV guest > >> > > must be in MFN space. > >> > > > >> > > >> > That's not a good approach. Dirvers reside on a higher level from = the > >> > "Xen specific bit". Doing this kind of translation will break thei= r > >> > generality. But there seems no other way to get this job done... > >> > >> This is a general problem with operating systems: bus addresses are = not > >> always the same as CPU memory addresses (even though in the easy cas= e of > >> a simple x86 PC they often are). =A0This is why OSes already have > >> interfaces to allocate DMA-able memory and to translate between the > >> address that CPU will map and the one the peripheral should use. =A0= If > >> your driver's not using the right address-space to program its hardw= are > >> it is broken; the Xen PV model is just an extreme case of this. > > > > Indeed. > > > > When passing addresses to devices, drivers should use the DMA APIs to > > make sure that proper address conversions take place (see > > Documentation/DMA-API-HOWTO.txt). > > On Xen the swiotlb implementation takes care of doing pfn to mfn > > conversions as well. > > For example, to allocate a consistent buffer that is addressable by b= oth > > a device and the cpu, the correct API is dma_alloc_coherent, that > > returns two values: the virtual address which you can use to access i= t > > from the CPU and dma_handle which you pass to the device. > > >=20 > This is the first time to notice swiotlb TBH. It seems to be a good > start for me. >=20 > After skimming the code in $KERNEL/drivers/xen/swiotlb-xen.c, I have > further questions. >=20 > It seems that swiotlb is used for PCI passthrough in PV, I'm wondering > if paravirt drivers ever use it. And when running in Dom0. >=20 > > I can very well imagine that virtio drivers are not doing any of this > > because they didn't need to, but it would be probably OK to submit a > > patch to convert the drivers to use the proper APIs. > > >=20 > No, they are not doing this. >=20 > Do you mean that I replace every allocation/deallocation of virtio > drivers scatter buffer with DMA API? Use the pci_map_sg calls and follow it with pci_sync calls. The DMA API mentions how to do it properly. Countless drivers do it. >=20 > Then, with 'swiotlb=3Dforce' in pv guest kernel command line, these > DMA-API will call xen-swiotlb and get proper addresses? Correct. Thought you don't need to use 'swiotlb=3Dforce'. You can just do 'iommu=3Dsoft' and that will take care of it in PV case. In Dom0 case there is no need for this argument. >=20 > If I don't add 'swiotlb=3Dforce', which underlying implementation will > be used? And, which DMA-API will be used in hvm case? will this change > affect hvm guest? For HVM if the machine has less than 4GB it will use the no_dma implement= ation. Which is to say: phys_to_virt and virt_to_phys for all DMA calls. If you have more than 4GB (or iommu=3Dsoft) it will use the native SWIOTLB implementation. (And the native SWIOTLB can't properly do PFN -> MFN or vice versa). >=20 > Wei. >=20 > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel