Re: [PATCH v2 10/11] xen/arm: introduce phys/dma translations in xen_dma_sync_for_*

Stefano Stabellini
2020-06-09T00:38:37Z

On Mon, 8 Jun 2020, Christoph Hellwig wrote:
> On Wed, Jun 03, 2020 at 03:22:46PM -0700, Stefano Stabellini wrote:
> > From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> >
> > xen_dma_sync_for_cpu, xen_dma_sync_for_device, xen_arch_need_swiotlb are
> > getting called passing dma addresses. On some platforms dma addresses
> > could be different from physical addresses. Before doing any operations
> > on these addresses we need to convert them back to physical addresses
> > using dma_to_phys.
> >
> > Add dma_to_phys calls to xen_dma_sync_for_cpu, xen_dma_sync_for_device,
> > and xen_arch_need_swiotlb.
> >
> > dma_cache_maint is fixed by the next patch.
>
> The calling conventions because really weird now because
> xen_dma_sync_for_{device,cpu} already get both a phys_addr_t and
> a dma_addr_t.
>
> >
> > - if (pfn_valid(PFN_DOWN(handle)))
> > + if (pfn_valid(PFN_DOWN(dma_to_phys(dev, handle))))
>
> But here we translate the dma address to a phys addr
>
> > arch_sync_dma_for_cpu(paddr, size, dir);
>
> While this still uses the passed in paddr. I think the uses of
> addresses in this code really needs a major rethink.

Yeah, the pfn_valid check is a bit weird by definition because we are
using it to understand whether the address belong to us or to another
VM. To do the pfn_valid check we need to translate the dma address into
something the CPU understands, hence, the dma_to_phys call.

Why can't we use the already-provided paddr? Because paddr has been
translated twice:
1) from dma to maybe-foreign phys address (could be ours, or another VM)
2) from maybe-foreign address to local (using our local mapping of the foreign page)

In fact, it would be clearer if we had all three addresses as parameters
of xen_dma_sync_for_cpu: the dma address, the maybe-foreign physical
address (we tend to call it xenbus address, baddr), the local physical
address. Something like:

void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle,
phys_addr_t baddr, phys_addr_t paddr, size_t size,
enum dma_data_direction dir)
{
if (pfn_valid(baddr))
arch_sync_dma_for_cpu(paddr, size, dir);
else if (dir != DMA_TO_DEVICE)
dma_cache_maint(dev, handle, size, GNTTAB_CACHE_INVAL);
}