Re: Linux grant map/unmap improvement proposal (Draft B)

[David Vrabel <david.vrabel@citrix.com>]

On 13/10/14 17:43, Stefano Stabellini wrote:
> On Mon, 13 Oct 2014, David Vrabel wrote:
>> Grant mapping in the Linux kernel has a number of problems:
>>
[...]
> This is a very good summary of the issues we are currently having with
> Xen support in Linux. As such, I think I should add one that is missing
> from the list, but good to keep in mind. I should point out that I am
> not asking you to do anything about it at the moment.
> 
> 
> dma_ops.unmap_page and dma_ops.unmap_sg only pass dma addresses as arguments
> ----------------------------------------------------------------------------
> 
> The Linux dma_map_ops API consists of a number of functions that only
> provide the dma address of the dma request as argument, not the struct
> page or the physical address. For example unmap_page and unmap_sg.
> 
> For Xen PV guests the dma address is a machine address. If the machine
> address corresponds to a foreign page (granted to the current domain),
> there is no easy way for us to retrieve the corresponding struct page or
> guest physical address (other than the m2p_override with all its
> problems).
> 
> This is a serious limitation, in particular if we need to do any
> operations on the memory region at the time one of these functions are
> called:
> - on x86 fortunately we don't need to do anything;
> - on ARM, if the device is not dma coherent, we might have to issue cache
> maintenance operations.

I can add this section but it would be even better with a solution for
the "Design" section.

>> Userspace address to page translation
>> -------------------------------------
>>
>> The m2p_override table shall be removed.
>>
>> Each VMA (struct vm_struct) shall contain an additional pointer to an
>> optional array of pages.  This array shall be sized to cover the full
>> extent of the VMA.
>>
>> The gntdev driver populates this array with the relevant pages for the
>> foreign mappings as they are mapped.  It shall also clear them when
>> unmapping.  The gntdev driver must ensure it properly splits the page
>> array when the VMA itself is split.
>>
>> Since the m2p lookup will not return a local PFN, the native
>> get_user_pages_fast() call will fail.  Prior to attempting to fault in
>> the pages, get_user_pages() can simply look up the pages in the VMA's
>> page array.
>>
>> `page->private` will no longer need to be set to the MFN.
>>
>> This is similar to the approach used in the classic kernel.
> 
> It is worth pointing out that if/when non dma coherent devices are going
> to start appearing in x86-land, this solution won't suffice.

It's not trying to solve that problem.

If non-coherent devices ever become a problem on x86 I would probably
extend the DMA API with map/unmap functions that return/accept handles
and update the drivers for the non-coherent devices to use the new
functions.

David