Re: [PATCH 0/2] KVM: MMU: support VMAs that got remap_pfn_range-ed

[Xiao Guangrong <guangrong.xiao@linux.intel.com>]

On 07/05/2016 01:16 PM, Neo Jia wrote:
> On Tue, Jul 05, 2016 at 12:02:42PM +0800, Xiao Guangrong wrote:
>>
>>
>> On 07/05/2016 09:35 AM, Neo Jia wrote:
>>> On Tue, Jul 05, 2016 at 09:19:40AM +0800, Xiao Guangrong wrote:
>>>>
>>>>
>>>> On 07/04/2016 11:33 PM, Neo Jia wrote:
>>>>
>>>>>>>
>>>>>>> Sorry, I think I misread the "allocation" as "mapping". We only delay the
>>>>>>> cpu mapping, not the allocation.
>>>>>>
>>>>>> So how to understand your statement:
>>>>>> "at that moment nobody has any knowledge about how the physical mmio gets virtualized"
>>>>>>
>>>>>> The resource, physical MMIO region, has been allocated, why we do not know the physical
>>>>>> address mapped to the VM?
>>>>>>
>>>>>
>>>>> >From a device driver point of view, the physical mmio region never gets allocated until
>>>>> the corresponding resource is requested by clients and granted by the mediated device driver.
>>>>
>>>> Hmm... but you told me that you did not delay the allocation. :(
>>>
>>> Hi Guangrong,
>>>
>>> The allocation here is the allocation of device resource, and the only way to
>>> access that kind of device resource is via a mmio region of some pages there.
>>>
>>> For example, if VM needs resource A, and the only way to access resource A is
>>> via some kind of device memory at mmio address X.
>>>
>>> So, we never defer the allocation request during runtime, we just setup the
>>> CPU mapping later when it actually gets accessed.
>>>
>>>>
>>>> So it returns to my original question: why not allocate the physical mmio region in mmap()?
>>>>
>>>
>>> Without running anything inside the VM, how do you know how the hw resource gets
>>> allocated, therefore no knowledge of the use of mmio region.
>>
>> The allocation and mapping can be two independent processes:
>> - the first process is just allocation. The MMIO region is allocated from physical
>>    hardware and this region is mapped into _QEMU's_ arbitrary virtual address by mmap().
>>    At this time, VM can not actually use this resource.
>>
>> - the second process is mapping. When VM enable this region, e.g, it enables the
>>    PCI BAR, then QEMU maps its virtual address returned by mmap() to VM's physical
>>    memory. After that, VM can access this region.
>>
>> The second process is completed handled in userspace, that means, the mediated
>> device driver needn't care how the resource is mapped into VM.
>
> In your example, you are still picturing it as VFIO direct assign, but the solution we are
> talking here is mediated passthru via VFIO framework to virtualize DMA devices without SR-IOV.
>

Please see my comments below.

> (Just for completeness, if you really want to use a device in above example as
> VFIO passthru, the second step is not completely handled in userspace, it is actually the guest
> driver who will allocate and setup the proper hw resource which will later ready
> for you to access via some mmio pages.)

Hmm... i always treat the VM as userspace.

>
>>
>> This is how QEMU/VFIO currently works, could you please tell me the special points
>> of your solution comparing with current QEMU/VFIO and why current model can not fit
>> your requirement? So that we can better understand your scenario?
>
> The scenario I am describing here is mediated passthru case, but what you are
> describing here (more or less) is VFIO direct assigned case. It is different in several
> areas, but major difference related to this topic here is:
>
> 1) In VFIO direct assigned case, the device (and its resource) is completely owned by the VM
> therefore its mmio region can be mapped directly into the VM during the VFIO mmap() call as
> there is no resource sharing among VMs and there is no mediated device driver on
> the host to manage such resource, so it is completely owned by the guest.

I understand this difference, However, as you told to me that the MMIO region allocated for the
VM is continuous, so i assume the portion of physical MMIO region is completely owned by guest.
The only difference i can see is mediated device driver need to allocate that region.

>
> 2) In mediated passthru case, multiple VMs are sharing the same physical device, so how
> the HW resource gets allocated is completely decided by the guest and host device driver of
> the virtualized DMA device, here is the GPU, same as the MMIO pages used to access those Hw resource.

I can not see what guest's affair is here, look at your code, you cooked the fault handler like
this:

+               ret = parent->ops->validate_map_request(mdev, virtaddr,
+                                                        &pgoff, &req_size,
+                                                        &pg_prot);

Please tell me what information is got from guest? All these info can be found at the time of
mmap().