Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device iotlb descriptor

[Jason Wang <jasowang@redhat.com>]

On 2017年02月20日 17:13, Liu, Yi L wrote:
>> -----Original Message-----
>> From: Jason Wang [mailto:jasowang@redhat.com]
>> Sent: Monday, February 20, 2017 5:04 PM
>> To: Liu, Yi L <yi.l.liu@intel.com>; Michael S. Tsirkin <mst@redhat.com>; qemu-
>> devel@nongnu.org
>> Cc: Lan, Tianyu <tianyu.lan@intel.com>; Peter Maydell
>> <peter.maydell@linaro.org>; Tian, Kevin <kevin.tian@intel.com>; Eduardo
>> Habkost <ehabkost@redhat.com>; Peter Xu <peterx@redhat.com>; Alex
>> Williamson <alex.williamson@redhat.com>; Paolo Bonzini
>> <pbonzini@redhat.com>; Richard Henderson <rth@twiddle.net>
>> Subject: Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device iotlb
>> descriptor
>>
>>
>>
>> On 2017年02月20日 16:27, Liu, Yi L wrote:
>>>> -----Original Message-----
>>>> From: Jason Wang [mailto:jasowang@redhat.com]
>>>> Sent: Friday, February 17, 2017 2:43 PM
>>>> To: Liu, Yi L <yi.l.liu@intel.com>; Michael S. Tsirkin
>>>> <mst@redhat.com>; qemu- devel@nongnu.org
>>>> Cc: Peter Maydell <peter.maydell@linaro.org>; Eduardo Habkost
>>>> <ehabkost@redhat.com>; Peter Xu <peterx@redhat.com>; Paolo Bonzini
>>>> <pbonzini@redhat.com>; Richard Henderson <rth@twiddle.net>; Tian,
>>>> Kevin <kevin.tian@intel.com>; Lan, Tianyu <tianyu.lan@intel.com>;
>>>> Alex Williamson <alex.williamson@redhat.com>
>>>> Subject: Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device
>>>> iotlb descriptor
>>>>
>>>>
>>>>
>>>> On 2017年02月17日 14:18, Liu, Yi L wrote:
>>>>>> -----Original Message-----
>>>>>> From: Jason Wang [mailto:jasowang@redhat.com]
>>>>>> Sent: Thursday, February 16, 2017 1:44 PM
>>>>>> To: Liu, Yi L <yi.l.liu@intel.com>; Michael S. Tsirkin
>>>>>> <mst@redhat.com>; qemu- devel@nongnu.org
>>>>>> Cc: Peter Maydell <peter.maydell@linaro.org>; Eduardo Habkost
>>>>>> <ehabkost@redhat.com>; Peter Xu <peterx@redhat.com>; Paolo Bonzini
>>>>>> <pbonzini@redhat.com>; Richard Henderson <rth@twiddle.net>; Tian,
>>>>>> Kevin <kevin.tian@intel.com>; Lan, Tianyu <tianyu.lan@intel.com>;
>>>>>> Alex Williamson <alex.williamson@redhat.com>
>>>>>> Subject: Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device
>>>>>> iotlb descriptor
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2017年02月16日 13:36, Liu, Yi L wrote:
>>>>>>>> -----Original Message-----
>>>>>>>> From: Qemu-devel
>>>>>>>> [mailto:qemu-devel-bounces+yi.l.liu=intel.com@nongnu.org]
>>>>>>>> On Behalf Of Michael S. Tsirkin
>>>>>>>> Sent: Tuesday, January 10, 2017 1:40 PM
>>>>>>>> To: qemu-devel@nongnu.org
>>>>>>>> Cc: Peter Maydell <peter.maydell@linaro.org>; Eduardo Habkost
>>>>>>>> <ehabkost@redhat.com>; Jason Wang <jasowang@redhat.com>; Peter
>>>> Xu
>>>>>>>> <peterx@redhat.com>; Paolo Bonzini <pbonzini@redhat.com>; Richard
>>>>>>>> Henderson <rth@twiddle.net>
>>>>>>>> Subject: [Qemu-devel] [PULL 08/41] intel_iommu: support device
>>>>>>>> iotlb descriptor
>>>>>>>>
>>>>>>>> From: Jason Wang <jasowang@redhat.com>
>>>>>>>>
>>>>>>>> This patch enables device IOTLB support for intel iommu. The
>>>>>>>> major work is to implement QI device IOTLB descriptor processing
>>>>>>>> and notify the device through iommu notifier.
>>>>>>>>
>>>>>>> Hi Jason/Michael,
>>>>>>>
>>>>>>> Recently Peter Xu's patch also touched intel-iommu emulation. His
>>>>>>> patch shadows second-level page table by capturing iotlb flush
>>>>>>> from guest. It would result in page table updating in host. Does
>>>>>>> this patch also use the same map/umap API provided by VFIO?
>>>>>> Yes, it depends on the iommu notifier too.
>>>>>>
>>>>>>> If it is, then I think it would also update page table in host. It
>>>>>>> looks to be a duplicate update. Pls refer to the following
>>>>>>> snapshot captured from section 6.5.2.5 of vtd spec.
>>>>>>>
>>>>>>> "Since translation requests from a device may be serviced by
>>>>>>> hardware from the IOTLB, software must always request IOTLB
>>>>>>> invalidation
>>>>>>> (iotlb_inv_dsc) before requesting corresponding Device-TLB
>>>>>>> (dev_tlb_inv_dsc) invalidation."
>>>>>>>
>>>>>>> Maybe for device-iotlb, we need a separate API which just pass
>>>>>>> down the invalidate info without updating page table. Any thoughts?
>>>>>> cc Alex.
>>>>>>
>>>>>> If we want ATS to be visible for guest (but I'm not sure if VFIO
>>>>>> support this), we probably need another notifier or a new flag.
>>>>> Jason, for assigned device, I think guest could see ATS if the
>>>>> assigned device supports ATS. I can see it when passthru iGPU.
>>>>>
>>>>> Regards,
>>>>> Yi L
>>>> Good to know this.
>>>>
>>>> If I understand your suggestion correctly, you want a dedicated API
>>>> to flush a hardware device IOTLB. I'm not sure this is really needed.
>>> yes, I'd like to have an extra API besides the current MAP/UNMAP.
>> I'm think whether or not we can do this without extra API or even don't need to
>> care about this.
>>
>>>> There's some discussion of similar issue in the past (when ATS is
>>>> used for virtio- net/vhost), looks like we could solve this by not
>>>> trigger the UNMAP notifier unless it was device IOTLB inv desc if ATS
>>>> is enabled for the device? With this remote IOMMU/IOTLB can only get
>>>> invalidation request once. For VFIO, the under layer IOMMU can deal
>>>> with hardware device IOTLB without any extra efforts.
>>> If I catch the background, I think it should be "not trigger the UNMAP
>>> notifier when unless it was device IOTLB inv desc if ATS is enabled for the
>> device"
>>
>> Seems not :)
>>
>> I mean, if ATS is enabled for the device, only trigger UNMAP notifier when
>> processing device IOTLB. Then we can only have flush once. And host IOMMU
>> driver will take care of device IOTLB flush too.
> hmmm, how about the iotlb inv desc which is prior to device-iotlb?

Any issue in this case?

> I'm not sure
> if it is practical to ignore the iotlb inv des since there is no SID info in it.

Yes, this needs some changes maybe.

Thanks

>
> Regards,
> Yi L