Re: [PATCH 1/1] iommu/vt-d: Prevent boot failure with devices requiring ATS

[Ethan Zhao <haifeng.zhao@linux.intel.com>]

On 9/4/2024 3:49 PM, Baolu Lu wrote:
> On 2024/9/4 14:49, Tian, Kevin wrote:
>>> From: Lu Baolu <baolu.lu@linux.intel.com>
>>> Sent: Wednesday, September 4, 2024 2:07 PM
>>>
>>> SOC-integrated devices on some platforms require their PCI ATS enabled
>>> for operation when the IOMMU is in scalable mode. Those devices are
>>> reported via ACPI/SATC table with the ATC_REQUIRED bit set in the Flags
>>> field.
>>>
>>> The PCI subsystem offers the 'pci=noats' kernel command to disable PCI
>>> ATS on all devices. Using 'pci=noat' with devices that require PCI ATS
>>> can cause a conflict, leading to boot failure, especially if the device
>>> is a graphics device.
>>>
>>> To prevent this issue, check PCI ATS support before enumerating the 
>>> IOMMU
>>> devices. If any device requires PCI ATS, but PCI ATS is disabled by
>>> 'pci=noats', switch the IOMMU to operate in legacy mode to ensure
>>> successful booting.
>>
>> I guess the reason of switching to legacy mode is because the platform
>> automatically enables ATS in this mode, as the comment says in
>> dmar_ats_supported(). This should be explained otherwise it's unclear
>> why switching the mode can make ATS working for those devices.
>
> Not 'automatically enable ATS,' but hardware provides something that is
> equivalent to PCI ATS. The ATS capability on the device is still
> disabled. That's the reason why such device must be an SOC-integrated
> one.

That is confusing, how to know the "hardware provides something that is
equivalent to PCI ATS" ? any public docs to say that ?

>
>>
>> But then doesn't it break the meaning of 'pci=noats' which means
>> disabling ATS physically? It's described as "do not use PCIe ATS and
>> IOMMU device IOTLB" in kernel doc, which is not equivalent to
>> "leave PCIe ATS to be managed by HW".
>
> Therefore, the PCI ATS is not used and the syntax of pci=noats is not
> broken.
>
>> and why would one want to use 'pci=noats' on a platform which
>> requires ats?
>
> We don't recommend users to disable ATS on a platform which has devices
> that rely on it. But nothing can prevent users from doing so. I am not
> sure why it is needed. One possible reason that I can think of is about
> security. Sometimes, people don't trust ATS because it allows devices to
> access the memory with translated requests directly without any
> permission check on the IOMMU end.

Appears that would happen with CXL link, while PCI link still will do
some checking (per VT-d spec sec 4.2.4). I have question here, such behaviour
happens with HW passthrough, also does to software passthrough (removed identity
mapping) ?


Thanks,
Ethan

>
> Thanks,
> baolu
>