Re: Is there a way to map pv guest pseudo physical address into dom0?

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Mon, Jun 27, 2011 at 09:35:24PM +0800, Wei Liu wrote:
> Thank you, Tim and Stefano.
> 
> On Mon, Jun 27, 2011 at 7:23 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Mon, 27 Jun 2011, Tim Deegan wrote:
> >> At 17:26 +0800 on 27 Jun (1309195596), Wei Liu wrote:
> >> > > You need to arrange for the Xen specific bit of the virtio frontend to
> >> > > do the p2m translation. All externally visible addresses from a PV guest
> >> > > must be in MFN space.
> >> > >
> >> >
> >> > That's not a good approach. Dirvers reside on a higher level from the
> >> > "Xen specific bit". Doing this kind of translation will break their
> >> > generality. But there seems no other way to get this job done...
> >>
> >> This is a general problem with operating systems: bus addresses are not
> >> always the same as CPU memory addresses (even though in the easy case of
> >> a simple x86 PC they often are).  This is why OSes already have
> >> interfaces to allocate DMA-able memory and to translate between the
> >> address that CPU will map and the one the peripheral should use.  If
> >> your driver's not using the right address-space to program its hardware
> >> it is broken; the Xen PV model is just an extreme case of this.
> >
> > Indeed.
> >
> > When passing addresses to devices, drivers should use the DMA APIs to
> > make sure that proper address conversions take place (see
> > Documentation/DMA-API-HOWTO.txt).
> > On Xen the swiotlb implementation takes care of doing pfn to mfn
> > conversions as well.
> > For example, to allocate a consistent buffer that is addressable by both
> > a device and the cpu, the correct API is dma_alloc_coherent, that
> > returns two values: the virtual address which you can use to access it
> > from the CPU and dma_handle which you pass to the device.
> >
> 
> This is the first time to notice swiotlb TBH. It seems to be a good
> start for me.
> 
> After skimming the code in $KERNEL/drivers/xen/swiotlb-xen.c, I have
> further questions.
> 
> It seems that swiotlb is used for PCI passthrough in PV, I'm wondering
> if paravirt drivers ever use it.

And when running in Dom0.
> 
> > I can very well imagine that virtio drivers are not doing any of this
> > because they didn't need to, but it would be probably OK to submit a
> > patch to convert the drivers to use the proper APIs.
> >
> 
> No, they are not doing this.
> 
> Do you mean that I replace every allocation/deallocation of virtio
> drivers scatter buffer with DMA API?

Use the pci_map_sg calls and follow it with pci_sync calls.
The DMA API mentions how to do it properly. Countless drivers do it.

> 
> Then, with 'swiotlb=force' in pv guest kernel command line, these
> DMA-API will call xen-swiotlb and get proper addresses?

Correct. Thought you don't need to use 'swiotlb=force'. You can
just do 'iommu=soft' and that will take care of it in PV case.

In Dom0 case there is no need for this argument.
> 
> If I don't add 'swiotlb=force', which underlying implementation will
> be used? And, which DMA-API will be used in hvm case? will this change
> affect hvm guest?

For HVM if the machine has less than 4GB it will use the no_dma implementation.
Which is to say: phys_to_virt and virt_to_phys for all DMA calls.

If you have more than 4GB (or iommu=soft) it will use the native
SWIOTLB implementation.

(And the native SWIOTLB can't properly do PFN -> MFN or vice versa).
> 
> Wei.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel