Re: [Qemu-devel] Re: [PATCH 2 of 5] add can_dma/post_dma for direct IO

[Anthony Liguori <anthony@codemonkey.ws>]

Avi Kivity wrote:
> Anthony Liguori wrote:
>>>
>>> I think any API based on a can_dma abstraction is wrong.  The 
>>> write_post_dma thing is equally wrong.
>>>
>>> The concept of "dma" that you're introducing is not correct.
>>>
>>> The DMA API should have the following properties:
>>>
>>> 1) You attempt to map a physical address.  This effectively is a 
>>> lock or pin operation.
>>>  a) In the process of this, you get a virtual address that you can 
>>> manipulate.
>>> 2) You do your IO to the virtual address
>>> 3) You indicate how much of the memory you have dirtied
>>> 4) You unmap or unlock that memory region.  The virtual address is 
>>> now no longer valid.
>>>
>>> This could correspond to a:
>>>
>>> void *cpu_physical_memory_map(target_phys_addr_t addr, ram_addr_t 
>>> size, int is_write);
>>>
>>> void cpu_physical_memory_unmap(target_physical_addr_t addr, 
>>> ram_addr_t size, void *mapping, int is_dirty);
>>
>> Let me clarify this a bit more.  The problem we're trying to address 
>> today is the encapsulating knowledge of phys_ram_base.  We want to 
>> minimize the amount of code that makes any assumptions about 
>> phys_ram_base.  Your current API still accesses phys_ram_base 
>> directly in the PCI DMA API.  The only real improvement compared to 
>> the current virtio code is that you properly handle MMIO.  This is 
>> not just about layout but this also includes the fact that in the 
>> future, guest memory could be discontiguous in QEMU (think memory 
>> hotplug).
>
> There are two more problems the dma api solves:
>
> - DMA into mmio regions; this requires bouncing

The map() API I proposed above should do bouncing to MMIO regions.  To 
deal with unbounded allocation, you can simply fail when the mapping 
allocation has reached some high limit.  Calling code needs to cope with 
the fact that map'ing may succeed or fail.

> - DMA with an associated transform (xor, byteswap); also requires 
> bouncing

At this layer of the API, this is unnecessary.  At the PCI layer, you 
would need to handle this and I'd suggest taking the same approach as above.

> In turn, bouncing requires splitting large requests to avoid unbounded 
> memory allocation.
>
> While I think _map/_unmap is an improvement over can_dma(), this API 
> can't handle bounded bouncing, and so a separate layer (dma.c) is 
> still necessary.

I think it can handled bounded bouncing fine.  It's a matter of ensuring 
the bounce buffer max is sufficiently large and ensuring that any client 
code can cope with map failures.  In fact, there probably needs to be a 
notifiers API that is invoked whenever an unmap() happens so that if an 
asynchronous request is waiting because of a map failure, it can be 
notified as to when it should try again.

Regards,

Anthony Liguori