Re: [PATCH v3 0/8] vfio/mdev: IOMMU aware mediated device

[Xu Zaibo <xuzaibo@huawei.com>]

Hi,

On 2018/10/16 9:21, Lu Baolu wrote:
> Hi,
>
> On 10/15/2018 04:50 PM, Xu Zaibo wrote:
>> Hi,
>>
>> On 2018/10/15 10:48, Lu Baolu wrote:
>>> Hi,
>>>
>>> On 10/13/2018 04:25 PM, Xu Zaibo wrote:
>>>> Hi,
>>>>
>>>> On 2018/10/12 13:16, Lu Baolu wrote:
>>>>> Hi,
>>>>>
>>>>> The Mediate Device is a framework for fine-grained physical device
>>>>> sharing across the isolated domains. Currently the mdev framework
>>>>> is designed to be independent of the platform IOMMU support. As the
>>>>> result, the DMA isolation relies on the mdev parent device in a
>>>>> vendor specific way.
>>>>>
>>>>> There are several cases where a mediated device could be protected
>>>>> and isolated by the platform IOMMU. For example, Intel vt-d rev3.0
>>>>> [1] introduces a new translation mode called 'scalable mode', which
>>>>> enables PASID-granular translations. The vt-d scalable mode is the
>>>>> key ingredient for Scalable I/O Virtualization [2] [3] which allows
>>>>> sharing a device in minimal possible granularity (ADI - Assignable
>>>>> Device Interface).
>>>>>
>>>>> A mediated device backed by an ADI could be protected and isolated
>>>>> by the IOMMU since 1) the parent device supports tagging an unique
>>>>> PASID to all DMA traffic out of the mediated device; and 2) the DMA
>>>>> translation unit (IOMMU) supports the PASID granular translation.
>>>>> We can apply IOMMU protection and isolation to this kind of devices
>>>>> just as what we are doing with an assignable PCI device.
>>>>>
>>>>> In order to distinguish the IOMMU-capable mediated devices from those
>>>>> which still need to rely on parent devices, this patch set adds two
>>>>> new members in struct mdev_device.
>>>>>
>>>>> * iommu_device
>>>>>    - This, if set, indicates that the mediated device could
>>>>>      be fully isolated and protected by IOMMU via attaching
>>>>>      an iommu domain to this device. If empty, it indicates
>>>>>      using vendor defined isolation.
>>>>>
>>>>> * iommu_domain
>>>>>    - This is a place holder for an iommu domain. A domain
>>>>>      could be store here for later use once it has been
>>>>>      attached to the iommu_device of this mdev.
>>>>>
>>>>> Below helpers are added to set and get above iommu device
>>>>> and iommu domain pointers in mdev core implementation.
>>>>>
>>>>> * mdev_set/get_iommu_device(dev, iommu_device)
>>>>>    - Set or get the iommu device which represents this mdev
>>>>>      in IOMMU's device scope. Drivers don't need to set the
>>>>>      iommu device if it uses vendor defined isolation.
>>>>>
>>>>> * mdev_set/get_iommu_domain(domain)
>>>>>    - A iommu domain which has been attached to the iommu
>>>>>      device in order to protect and isolate the mediated
>>>>>      device will be kept in the mdev data structure and
>>>>>      could be retrieved later.
>>>>>
>>>>> The mdev parent device driver could opt-in that the mdev could be
>>>>> fully isolated and protected by the IOMMU when the mdev is being
>>>>> created by invoking mdev_set_iommu_device() in its @create().
>>>> I just cannot understand here, how to get an iommu_device while I 
>>>> create mediated
>>>> device in my parent device driver?
>>>
>>> When you are creating an mdev in your parent driver, you should know
>>> which PCI device this mdev belonging to.
>>>
>>
>> So, generally, I can set the parent device as mdev's iommu_device?
>> If that, however, Mdev already holds its parent device. So, I just 
>> figure what
>> differences between Mdev's parent device and iommu_device are.
>>>>
>>>> And why not reuse the device of MDEV instread of adding a new 
>>>> device here?
>>>
>>> iommu_device in the mdev_device structure represents the PCI device
>>> that represents this mdev in iommu's device scope. IOMMU is only aware
>>> of pci devices, it's not aware of mdev device.
>>
>> Could I understand like that: IOMMU can be aware of the parent device 
>> of Mdev?
>> And more, I am doubting the necessary of iommu_device in Mdev.
>>
>
> The "mdev parent device" and "mdev iommu device" are different although
> they might be the same in practice. "mdev parent device" represents the
> device who created the mdev. "mdev iommu device" represents the device
> who shares the device context entry in iommu tables.
>
> "mdev iommu device" is always a PCI/PCIe device since IOMMU always use
> source id (bus:dev:func) to walk the device context table. But there is
> no limitation on who can create an mdev, right?
>
Actually, I am not sure.

My understanding:
The DMA address will be issued by the parent device with PASID or 
something like that to IOMMU
facilities. However, the translation units such as iommu （PASID/page 
.etx）tables are from another
device node.  I cannot figure out how to control this in hardware level, 
or whether there will be
conflicts between the DMA transation of iommu_device and parent device.

Thanks,
Zaibo

.