Re: [PATCH v7 9/9] iommu/vt-d: Add iotlb flush for nested domain

Yi Liu
2023-12-26T04:49:04Z

On 2023/12/22 14:57, Tian, Kevin wrote:
>> From: Liu, Yi L <yi.l.liu@intel.com>
>> Sent: Thursday, December 21, 2023 11:40 PM
>>
>> +
>> +static void intel_nested_flush_cache(struct dmar_domain *domain, u64
>> addr,
>> + unsigned long npages, u32 *error)
>> +{
>> + struct iommu_domain_info *info;
>> + unsigned long i;
>> + unsigned mask;
>> + u32 fault = 0;
>> +
>> + if (npages == U64_MAX)
>> + mask = 64 - VTD_PAGE_SHIFT;
>> + else
>> + mask = ilog2(__roundup_pow_of_two(npages));
>> +
>> + xa_for_each(&domain->iommu_array, i, info) {
>> + nested_flush_pasid_iotlb(info->iommu, domain, addr,
>> npages, 0);
>
> so IOMMU_VTD_INV_FLAGS_LEAF is defined but ignored?

yeah... it is. It is named as ih in the driver code. But it appears only
the below code is set ih. When calling iommu_flush_iotlb_psi(), the 5th
parameter (ih) may be true.

static int intel_iommu_memory_notifier(struct notifier_block *nb,
unsigned long val, void *v)
{
struct memory_notify *mhp = v;
unsigned long start_vpfn = mm_to_dma_pfn(mhp->start_pfn);
unsigned long last_vpfn = mm_to_dma_pfn(mhp->start_pfn +
mhp->nr_pages - 1);

switch (val) {
case MEM_GOING_ONLINE:
if (iommu_domain_identity_map(si_domain,
start_vpfn, last_vpfn)) {
pr_warn("Failed to build identity map for [%lx-%lx]\n",
start_vpfn, last_vpfn);
return NOTIFY_BAD;
}
break;

case MEM_OFFLINE:
case MEM_CANCEL_ONLINE:
{
struct dmar_drhd_unit *drhd;
struct intel_iommu *iommu;
LIST_HEAD(freelist);

domain_unmap(si_domain, start_vpfn, last_vpfn, &freelist);

rcu_read_lock();
for_each_active_iommu(iommu, drhd)
iommu_flush_iotlb_psi(iommu, si_domain,
start_vpfn, mhp->nr_pages,
list_empty(&freelist), 0);
rcu_read_unlock();
put_pages_list(&freelist);
}
break;
}

return NOTIFY_OK;
}

>
>> +
>> + if (domain->has_iotlb_device)
>> + continue;
>> +
>> + nested_flush_dev_iotlb(domain, addr, mask, &fault);
>> + if (fault & (DMA_FSTS_ITE | DMA_FSTS_ICE))
>> + break;
>
> here you may add a note that we don't plan to forward invalidation
> queue error (i.e. IQE) to the caller as it's caused only by driver
> internal bug.

yes.

>
>> +
>> + if (!IS_ALIGNED(inv_entry.addr, VTD_PAGE_SIZE) ||
>> + ((inv_entry.npages == U64_MAX) && inv_entry.addr)) {
>> + ret = -EINVAL;
>> + break;
>> + }
>> +
>
> why is [non-zero-addr, U64_MAX] an error? Is it explicitly stated to
> be not supported by underlying helpers?

no such limitation by underlying helpers. But in such case, the
addr+npages*PAGE_SIZE would exceed U64_MAX, this seems a bit
strange. But I'm fine to relax the check since the underlying helper
only checks npages when determining paid-selective or not.

--
Regards,
Yi Liu