Re: [PATCH] hw/virtio/vhost: Disable IOTLB callbacks when IOMMU gets disabled

[Eric Auger <eric.auger@redhat.com>]

Hi Zhenzhong,


On 1/21/25 10:18 AM, Duan, Zhenzhong wrote:
> Hi Eric,
>
>> -----Original Message-----
>> From: Eric Auger <eric.auger@redhat.com>
>> Subject: [PATCH] hw/virtio/vhost: Disable IOTLB callbacks when IOMMU gets
>> disabled
>>
>> When a guest exposed with a vhost device and protected by an
>> intel IOMMU gets rebooted, we sometimes observe a spurious warning:
>>
>> Fail to lookup the translated address ffffe000
> Do you see this print once during one time reboot?
Actually this happens rarely on reboot. The reproducibility is of the
order of magnitude of 1/10 for me. I use a vm with vhost net device +
virtual intel iommu featuring a crontab job.
@reboot /usr/sbin/reboot
>
>> We observe that the IOMMU gets disabled through a write to the global
>> command register (CMAR_GCMD.TE) before the vhost device gets stopped.
>> When this warning happens it can be observed an inflight IOTLB
>> miss occurs after the IOMMU disable and before the vhost stop. In
>> that case a flat translation occurs and the check in
>> vhost_memory_region_lookup() fails.
>>
>> Let's disable the IOTLB callbacks when all IOMMU MRs have been
>> unregistered.
> Try to understand the sequence, is it like below?
>
>            vhost                                                     vcpu          
>
> call into vtd_iommu_translate();
No that's a kernel vhost translate request that normally tries to find
out the translated address on kernel side in the IOTLB but since the
data is not there it ends up asking for the translation to user space ...
>
>                                                                     set s->dmar_enabled = false;
>                                                                     switch off iommu address space;
>                                                                     disable vhost IOTLB callbacks;
vtd_handle_gcmd_write/vtd_handle_gcmd_te/vtd_handle_gcmd_te which
eventually calls vhost_iommu_region_del
>
> check if !s->dmar_enabled;
> return flat translation and trigger warning
vhost inflight translation reaches user space through
vhost_device_iotlb_miss()
>
> Thanks
> Zhenzhong
>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>> ---
>> hw/virtio/vhost.c | 4 ++++
>> 1 file changed, 4 insertions(+)
>>
>> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
>> index 6aa72fd434..128c2ab094 100644
>> --- a/hw/virtio/vhost.c
>> +++ b/hw/virtio/vhost.c
>> @@ -931,6 +931,10 @@ static void vhost_iommu_region_del(MemoryListener
>> *listener,
>>             break;
>>         }
>>     }
>> +    if (QLIST_EMPTY(&dev->iommu_list) &&
>> +        dev->vhost_ops->vhost_set_iotlb_callback) {
>> +        dev->vhost_ops->vhost_set_iotlb_callback(dev, false);
>> +    }
>> }
>>
>> void vhost_toggle_device_iotlb(VirtIODevice *vdev)
>> --
>> 2.47.1