Re: [PATCH] vhost: Unbreak SMMU and virtio-iommu on dev-iotlb support

[Auger Eric <eric.auger@redhat.com>]

Hi,

On 2/7/21 3:47 PM, Peter Xu wrote:
> Hi, Kevin,
> 
> On Sun, Feb 07, 2021 at 09:04:55AM +0000, Tian, Kevin wrote:
>>> From: Peter Xu
>>> Sent: Friday, February 5, 2021 11:31 PM
>>>
>>>>>
>>>>>
>>>>>> or virtio-iommu
>>>>>> since dev-iotlb (or PCIe ATS)
>>>>>
>>>>>
>>>>> We may need to add this in the future.
>>>> added Jean-Philippe in CC
>>>
>>> So that's the part I'm unsure about..  Since everybody is cced so maybe good
>>> time to ask. :)
>>>
>>> The thing is I'm still not clear on whether dev-iotlb is useful for a full
>>> emulation environment and how that should differ from a normal iotlb, since
>>> after all normal iotlb will be attached with device information too.
>>
>> dev-iotlb is useful in two manners.First, it's a functional prerequisite for
>> supporting I/O page faults.
If I understand correctly, the stall model of the ARM SMMU allows IOPF I
guess without dev-iotlb (ATS). However indeed PRI requires ATS.
> 
> Is this also a hard requirement for virtio-iommu, which is not a real hardware
> after all?
> 
>> Second, it has performance benefit as you don't
>> need to contend the lock of global iotlb.
> 
> Hmm.. are you talking about e.g. vt-d driver or virtio-iommu?
> 
> Assuming it's about vt-d, qi_flush_dev_iotlb() will still call qi_submit_sync()
> and taking the same global QI lock, as I see it, or I could be wrong somewhere.
> I don't see where dev-iotlb has a standalone channel for delivery.
> 
> For virtio-iommu, we haven't defined dev-iotlb, right?
no there is no such feature at the moment. If my understanding is
correct this would only make sense when protecting a HW device. In that
case the underlying physical IOMMU would be programmed for ATS.

When protecting a virtio device (inc. vhost) what would be the adventage
over the current standard unmap notifier?

Thanks

Eric
  Sorry I missed things
> when I completely didn't follow virtio-iommu recently - let's say if
> virtio-iommu in the future can support per-dev dev-iotlb queue so it doesn't
> need a global lock, what if we make it still per-device but still delivering
> iotlb message?  Again, it's still a bit unclear to me why a full emulation
> iommu would need that definition of "iotlb" and "dev-iotlb".
> 
>>
>>>
>>> For real hardwares, they make sense because they ask for two things: iotlb is
>>> for IOMMU, but dev-iotlb is for the device cache.  For emulation
>>> environment
>>> (virtio-iommu is the case) do we really need that complexity?
>>>
>>> Note that even if there're assigned devices under virtio-iommu in the future,
>>> we can still isolate that and iiuc we can easily convert an iotlb (from
>>> virtio-iommu) into a hardware IOMMU dev-iotlb no matter what type of
>>> IOMMU is
>>> underneath the vIOMMU.
>>>
>>
>> Didn't get this point. Hardware dev-iotlb is updated by hardware (between
>> the device and the IOMMU). How could software convert a virtual iotlb
>> entry into hardware dev-iotlb?
> 
> I mean if virtio-iommu must be run in a guest, then we can trap that message
> first, right?  If there're assigned device in the guest, we must convert that
> invalidation to whatever message required for the host, that seems to not
> require the virtio-iommu to have dev-iotlb knowledge, still?
> 
> Thanks,
>