Re: [PATCH v6 09/14] iommu/vt-d: Add IOMMU_HWPT_ALLOC_PASID support

[Yi Liu <yi.l.liu@intel.com>]

On 2025/2/12 21:00, Robin Murphy wrote:
> On 2025-02-12 7:47 am, Yi Liu wrote:
>> On 2024/12/25 15:13, Baolu Lu wrote:
>>> On 2024/12/25 12:30, Yi Liu wrote:
>>>> On 2024/12/25 09:02, Baolu Lu wrote:
>>>>> On 12/24/24 19:35, Yi Liu wrote:
>>>>>>> Another related consideration is the support for page faults in nested
>>>>>>> domains once PASID is available in user space. Would it be 
>>>>>>> reasonable to
>>>>>>> support page faults for nested domains?
>>>>>>
>>>>>> yeah, it's good to discuss it.
>>>>>>
>>>>>>> If so, perhaps it's time to open intel_iommu_domain_alloc_nested() to
>>>>>>> support IOMMU_HWPT_FAULT_ID_VALID when PRI is enabled on the device?
>>>>>>
>>>>>> IMHO, the PRQ support for nested domains requires some more facility. 
>>>>>> PRQ
>>>>>> can happen at either stage-1 or stage-2, iommu driver may need to tell
>>>>>> it and forward to the correct domain (nested domain or parent 
>>>>>> domain). Or
>>>>>> the stage-2 is always pinned just as the VFIO/iommufd does. Hence, 
>>>>>> any PRQ
>>>>>> happens under nested translation should be due to stage-1.
>>>>>
>>>>> The parent domain is currently always pinned and does not yet support
>>>>> page faults. Therefore, when a page fault occurs within a nested domain,
>>>>> it apparently should be routed to user space...
>>>>>
>>>>> If we decide to support page faults on the stage-2 domain in the future,
>>>>> we will need to figure out the correct destination of each page fault
>>>>> and route it accordingly, either to the parent domain or the user space
>>>>> nested domain. Hardware assistance would be beneficial, otherwise the
>>>>> software may need to traverse the parent domain, which is not
>>>>> performance friendly.
>>>>
>>>> this is my question. Is it still true the parent domain is always pinned
>>>> after the below series? If yes, then it's fine to enable IOPF for nested
>>>> domain.
>>>>
>>>> https://lore.kernel.org/linux-iommu/20241015-jag-iopfv8-v4-0- 
>>>> b696ca89ba29@kernel.org/
>>>
>>> Then, perhaps we could enforce this in iommufd for a short-term purpose?
>>>
>>> When allocating a hwpt in iommufd, we should enforce that the flags
>>> IOMMU_HWPT_ALLOC_NEST_PARENT and IOMMU_HWPT_FAULT_ID_VALID cannot be set
>>> at the same time.
>>
>> Let's consult with other vendors. We need this enforcement because we lack
>> a straightforward method to distinguish between PRIs in stage-1 and stage-2
>> under nested translation. If other vendors share this requirement, it would
>> be appropriate to implement this enforcement in IOMMUFD. Otherwise, we may
>> check it in intel iommu driver.
>>
>> @arm and amd folks. :)
> 
> Yup, SMMUv3 is more or less in the same boat - we *could* reasonably manage 
> unpinned S2 for non-PCI devices using the stall model where the 
> F_TRANSLATION or F_PERMISSION event tells us all we need, but for ATS it's 
> the same thing where by the time the fault has taken a round-trip through 
> an ATS response and a PRI request, we've lost the details of exactly how it 
> faulted (or if the PRI request was sent eagerly without a prior translation 
> request, then we simply have no idea at all). Thus the prospective 
> mechanism would be to inject a virtual PRI, wait until we see the guest 
> issue a matching CMD_PRI_RESP, then sniff the IPA/GPA for the given input 
> address out of the guest pagetables to see if there's anything to do at S2 
> as well/instead. Yuck.

Thanks for the response, Robin. It seems we're in the same situation. :(
Out of curiosity, will the vPRI response provide the IPA/GPA to the
hypervisor? If not, the hypervisor will need to derive it from the GVA on
its own, possibly requiring software to traverse the guest page table.

-- 
Regards,
Yi Liu