Re: [PATCH v2 2/2] vring: Force use of DMA API for ARM-based systems

["Michael S. Tsirkin" <mst@redhat.com>]

On Mon, Jan 16, 2017 at 02:21:03PM +0000, Will Deacon wrote:
> On Mon, Jan 16, 2017 at 04:18:03PM +0200, Michael S. Tsirkin wrote:
> > On Mon, Jan 16, 2017 at 10:40:28AM +0000, Will Deacon wrote:
> > > On Fri, Jan 13, 2017 at 08:23:35PM +0200, Michael S. Tsirkin wrote:
> > > > On Fri, Jan 13, 2017 at 05:21:54PM +0000, Will Deacon wrote:
> > > > > On Fri, Jan 13, 2017 at 06:46:32PM +0200, Michael S. Tsirkin wrote:
> > > > > > On Fri, Jan 13, 2017 at 09:25:22AM +0000, Will Deacon wrote:
> > > > > > > On Fri, Jan 13, 2017 at 12:12:56AM +0200, Michael S. Tsirkin wrote:
> > > > > > > > I'd rather people didn't use SMMU with legacy devices.
> > > > > > > 
> > > > > > > I'm afraid we've been doing that for two years and the model already
> > > > > > > exists in a mature state, being actively used for development and
> > > > > > > validation by ARM and our partners. One of the big things its used for
> > > > > > > is to develop SMMU and GIC (our interrupt controller) code with PCI, so
> > > > > > > dropping the SMMU from the picture isn't an option.
> > > > > > 
> > > > > > Oh so this fixes a regression?  This is something I didn't realize.
> > > > > 
> > > > > Yes, thanks. The regression came about because we implemented SMMU-backed
> > > > > DMA ops and only then was it apparent that the virtio stuff was bypassing
> > > > > even with translation enabled (because it wasn't using the DMA API).
> > > > 
> > > > Could you point out a commit ID?
> > > 
> > > There has been a fair amount of work in this area recently, but you're
> > > probably after something like 876945dbf649 ("arm64: Hook up IOMMU dma_ops")
> > > as the culprit, which is the point at which we started to swizzle DMA
> > > ops for devices upstream of an SMMU automatically.
> > > 
> > > > > > A "Fixes:" tag can't hurt here.  I then wonder
> > > > > > might DMA ops ever use a DMA address which isn't a physical address
> > > > > > from QEMU point of view? If that happens, this hack breaks
> > > > > > because in legacy mode QEMU still uses the GPA.
> > > > > 
> > > > > If QEMU doesn't advertise an SMMU, then it will work fine with the GPA,
> > > > > because we won't swizzle the DMA ops for the master device. If QEMU does
> > > > > advertise an SMMU, then we'll allocate DMA addresses to fit within the
> > > > > the intersection of the SMMU aperture and device's DMA mask.
> > > > 
> > > > 
> > > > Right but doesn't just poking from qemu into phys addresses work
> > > > anymore? It used to ...
> > > 
> > > Provided that there's no SMMU, then it will continue to work. and my
> > > understanding (from talking to Peter Maydell) is that qemu doesn't model
> > > an SMMU for ARM-based machines.
> > > 
> > 
> > So how come people report failures due to presence of SMMU?
> > Using some other hypervisor?
> 
> The failures are reported on the ARM fastmodel (a complete system
> emulation that runs on an x86 box), where an SMMU *is* present
> downstream of the virtio-pci masters. There's no qemu involved there.
> 
> Will

I see. And this hypervisor actually coded up looking up
translations in the SMMU unconditionally for legacy devices,
and this worked as long as guest didn't touch the SMMU?

-- 
MST