Re: [PATCH v2 08/10] iommu/vt-d: Use the generic iommu page table

[Baolu Lu <baolu.lu@linux.intel.com>]

On 8/27/2025 1:26 AM, Jason Gunthorpe wrote:
> Replace the VT-D iommu_domain implementation of the VTD second stage and
> first stage page tables with the iommupt VTDSS and x86_64
> pagetables. x86_64 is shared with the AMD driver.
> 
> There are a couple notable things in VT-D:
> - Like AMD the second stage format is not sign extended, unlike AMD it
>    cannot decode a full 64 bits. The first stage format is a normal sign
>    extended x86 page table
> - The HW caps can indicate how many levels, how many address bits and what
>    leaf page sizes are supported in HW. As before the highest number of
>    levels that can translate the entire supported address width is used.
>    The supported page sizes are adjusted directly from the dedicated
>    first/second stage cap bits.
> - VTD requires flushing 'write buffers'. This logic is left unchanged,
>    the write buffer flushes on any gather flush or through iotlb_sync_map.
> - Like ARM, VTD has an optional non-coherent page table walker that
>    requires cache flushing. This is supported through PT_FEAT_DMA_INCOHERENT
>    the same as ARM, however x86 can't use the DMA API for flush, it must
>    call the arch function clflush_cache_range()
> - The PT_FEAT_DYNAMIC_TOP can probably be supported on VTD someday for the
>    second stage when it uses 128 bit atomic stores for the HW context
>    structures.
> - PT_FEAT_VTDSS_FORCE_WRITEABLE is used to work around ERRATA_772415_SPR17
> - A kernel command line parameter "sp_off" disables all page sizes except
>    4k
> 
> Remove all the unused iommu_domain page table code. The debugfs paths have
> their own independent page table walker that is left alone for now.
> 
> This corrects a race with the non-coherent walker that the ARM
> implementations have fixed:
> 
>       CPU 0                               CPU 1
>    pfn_to_dma_pte()                    pfn_to_dma_pte()
>     pte = &parent[offset];
>     if (!dma_pte_present(pte)) {
>       try_cmpxchg64(&pte->val)
> 					pte = &parent[offset];
> 					.. dma_pte_present(pte) ..
> 				        [...]
> 					// iommu_map() completes
> 					// Device does DMA
>       domain_flush_cache(pte)
> 
> The CPU 1 mapping operation shares a page table level with the CPU 0
> mapping operation. CPU 0 installed a new page table level but has not
> flushed it yet. CPU1 returns from iommu_map() and the device does DMA. The
> non coherent walker fails to see the new table level installed by CPU 0
> and fails the DMA with non-present.
> 
> The iommupt PT_FEAT_DMA_INCOHERENT implementation uses the ARM design of
> storing a flag when CPU 0 completes the flush. If the flag is not set CPU
> 1 will also flush to ensure the HW can fully walk to the PTE being
> installed.
> 
> Cc: Tina Zhang<tina.zhang@intel.com>
> Signed-off-by: Jason Gunthorpe<jgg@nvidia.com>
> ---
>   drivers/iommu/intel/Kconfig  |   4 +
>   drivers/iommu/intel/iommu.c  | 896 ++++++-----------------------------
>   drivers/iommu/intel/iommu.h  |  99 +---
>   drivers/iommu/intel/nested.c |   5 -
>   drivers/iommu/intel/pasid.c  |  29 +-
>   5 files changed, 175 insertions(+), 858 deletions(-)

Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>