Re: [PATCH 2/8] iommu/vt-d: Add entry_sync support for PASID entry updates

[Baolu Lu <baolu.lu@linux.intel.com>]

On 3/23/26 21:07, Jason Gunthorpe wrote:
> On Sun, Mar 15, 2026 at 04:11:36PM +0800, Baolu Lu wrote:
>>>>> ATC invalidations should always be done after the PASID entry is
>>>>> written. During a hitless update both translations are unpredictably
>>>>> combined, this is unavoidable and OK.
>>>>
>>>> The VT-d spec (Sections 6.5.2.5 and 6.5.2.6) explicitly mandates that an
>>>> IOTLB invalidation must precede the Device-TLB invalidation. If we only
>>>> do the device-TLB invalidation in the sync callback, we risk the device
>>>> re-fetching a stale translation from the IOMMU's internal IOTLB.
>>>
>>> It is a little weird that is says that, that is worth checking into.
>>>
>>> The other text is clear that the IOTLB is cached by DID,PASID only, so
>>> if the new PASID entry has a DID,PASID which is already coherent in
>>> the IOTLB it should not need any IOTLB flushing.
>>>
>>> ie flushing the PASID table should immediately change any ATC fetches
>>> from using DID,old_PASID to DID,new_PASID.
>>>
>>> If there is some issue where the PASID flush doesn't fence everything
>>> (ie an ATC fetch of DID,old_PASID can be passed by an ATC invalidation)
>>> then you may need IOTLB invalidations not to manage coherence but to
>>> manage ordering. That is an important detail if true.
>>
>> On Intel hardware, the PASID-cache and IOTLB are not inclusive. A PASID-
>> cache invalidation forces a re-fetch of the pasid entry, but it does not
>> automatically purge downstream IOTLB entries.
> 
> It doesn't matter, the updated PASID entry will point to a new DID and
> the IOTLB (new DID,PASID) entry will be valid in the IOTLB.
> 
> We don't need to flush the IOTLB, we just need to ensure that all
> lookups done with (old DID,PASID) are completed before sending any
> invalidation.

Yes, you are right.

> 
>> The spec-mandated IOTLB flush serves as a synchronization barrier to
>> ensure that in-flight translation requests are drained and the
>> internal IOMMU state is consistent before the invalidation request
>> is sent over PCIe to the device's ATC.
> 
> A fencing requirement does make sense, but does it have to be done by
> flushing the entire DID,PASID? It is ugly to have to drop the IOTLB
> just because a context entry changed.

I believe the full [old_DID, PASID] invalidation is a functional
necessity rather than just a fencing requirement. Even though the new
PASID entry points to a new_DID, leaving stale translations tagged with
[old_DID, PASID] in the IOTLB is problematic.

However, I agree that IOTLB and Device-TLB invalidation should not be
part of the entry_sync for a PASID entry; instead, it belongs in the
domain replacement logic.

> Can you do a 4k IOVA 0 invalidation and get the same fence?
> 
> Jason

Thanks,
baolu