Re: About unmap pages and set dirty tracking on nested parent domain

[Yi Liu <yi.l.liu@intel.com>]

On 2024/1/26 08:19, Jason Gunthorpe wrote:
> On Thu, Jan 25, 2024 at 10:03:31AM -0400, Jason Gunthorpe wrote:
>> On Thu, Jan 25, 2024 at 09:55:46PM +0800, Yi Liu wrote:
>>> Hi Jason, Kevin,
>>>
>>> Today, Intel iommu driver only tracks attached devices/iommus in the nested
>>> domain. While the nested parent domain does not.
>>
>> Heh, I was just looking at this bug on my ARM implemention too :)
> 
> What I did for SMMU is on my github how:
> 
> https://github.com/jgunthorpe/linux/commits/smmuv3_newapi/
> 
> See iommu/arm-smmu-v3: Support IOMMU_DOMAIN_NESTED

thanks for the sharing.

> The list that tracks where the iommu domain sends its invalidations
> gets a flag indicating that ATC has to be a full flush

yeah, I pass a 0 - MAX to do it.

> The domain gets a flag that says the IOTLB has to be a full flush (to
> wipe the nest child entries too)

yes

> Also note that if the VM is being relied on to generate ATC
> invalidations then the hypervisor must also track the VM's ATS
> state. If the VM thinks ATS is off then it will not issue ATC flushes.

for now, we don't rely on VM to issue ATC invalidations as we don't have
plan to forward the ATC invalidation timeout error to VM. As one of your
prior remarks (it should have be handled in a RAS arch). So we don't really
need VM to generate ATC invalidation since intel iommu driver will flush
ATC if host has enabled it. So I think VT-d side is fine on this so far.

> ARM has a convenient STE bit that makes this simple, otherwise the VMM
> will have to trap the PCI-E ATS config write and forward it through
> iommufd somehow.

yes. This may need new API for device driver (VFIO) to pass such info.

> Tracking all the ATS stuff was a big PITA, I think the approach I got
> for SMMU is pretty good though.

-- 
Regards,
Yi Liu