Re: [Qemu-devel] Re: [PATCH 2 of 5] add can_dma/post_dma for direct IO

Blue Swirl
2008-12-16T18:18:07Z

On 12/16/08, Anthony Liguori <anthony@codemonkey.ws> wrote:
> Avi Kivity wrote:
>
> > Blue Swirl wrote:
> >
> > >
> > > > I don't understand. It's not a device that needs bouncing, it's a
> > > > particular transfer. This could be either due to the transfer
> targeting
> > > > mmio, or due to the transfer requiring a transformation.
> > > >
> > > >
> > >
> > > Should the bouncing be something more much complex, for example
> > > negotiated between the devices? Or maybe the devices set up a
> > > transforming and non-transforming channel (which the other end should
> > > be able to transform some more) and use them as needed?
> > >
> > >
> >
> > Yes. We already have two cases:
> >
> > - may do partial request: useful for block storage where requests can be
> huge
> > - need full request: networking, where you can't send or receive half a
> packet; on the other hand, packets are small (even with tso)
> >
> > You're adding a third case, always bounce, when the data undergoes a
> transformation which can't happen in-place.
> >
>
> IMHO, IOMMU translation is distinct from mapping/data transformation. I
> would think the IOMMU translation API would be different in that it took a
> physical address and returned a physical address.

Fully agree. On Sparc32, the incoming address is called DVMA (device
virtual memory address).

> The IOMMU DMA API (which could transform data potentially) should return a
> virtual address and data a physical address. In the normal case
> (non-transforming IOMMU), it should just fall-through to CPU DMA API (which
> is just map/unmap).
>
> The PCI DMA API would normally fall through to the IOMMU DMA API.
>
> So we would have:
>
> PCI DMA map(addr):
> if IOMMU:
> return IOMMU DMA map(addr)
> return CPU DMA map(addr)
>
> IOMMU DMA map(addr):
> new_address = IOMMU translate(addr)
> if transform:
> return IOMMU byte-swap map(addr)
> return CPU DMA map(addr)

But with this we would be back to the original merged API.

I'd propose three to four distinct steps:
1. Resolve the device addresses via IOMMU etc. to guest physical
addresses (typically no changes if no IOMMU)
2. Negotiate bouncing (if bounced, the step returns a host address, if
not, a physical address)
3. Map the physical addresses to host addresses, bypass for the bounce cases
4. Perform vectorized zero-copy AIO. (Profit?)

2 & 3 may/should be merged but I'm not sure we have the full picture yet.