Re: [PATCH v6 09/14] iommu/vt-d: Add IOMMU_HWPT_ALLOC_PASID support

[Yi Liu <yi.l.liu@intel.com>]

On 2025/2/13 18:24, Robin Murphy wrote:
> On 2025-02-13 10:10 am, Yi Liu wrote:
>> On 2025/2/12 21:00, Robin Murphy wrote:
>>> On 2025-02-12 7:47 am, Yi Liu wrote:
>>>> On 2024/12/25 15:13, Baolu Lu wrote:
>>>>> On 2024/12/25 12:30, Yi Liu wrote:
>>>>>> On 2024/12/25 09:02, Baolu Lu wrote:
>>>>>>> On 12/24/24 19:35, Yi Liu wrote:
>>>>>>>>> Another related consideration is the support for page faults in 
>>>>>>>>> nested
>>>>>>>>> domains once PASID is available in user space. Would it be 
>>>>>>>>> reasonable to
>>>>>>>>> support page faults for nested domains?
>>>>>>>>
>>>>>>>> yeah, it's good to discuss it.
>>>>>>>>
>>>>>>>>> If so, perhaps it's time to open intel_iommu_domain_alloc_nested() to
>>>>>>>>> support IOMMU_HWPT_FAULT_ID_VALID when PRI is enabled on the device?
>>>>>>>>
>>>>>>>> IMHO, the PRQ support for nested domains requires some more 
>>>>>>>> facility. PRQ
>>>>>>>> can happen at either stage-1 or stage-2, iommu driver may need to tell
>>>>>>>> it and forward to the correct domain (nested domain or parent 
>>>>>>>> domain). Or
>>>>>>>> the stage-2 is always pinned just as the VFIO/iommufd does. Hence, 
>>>>>>>> any PRQ
>>>>>>>> happens under nested translation should be due to stage-1.
>>>>>>>
>>>>>>> The parent domain is currently always pinned and does not yet support
>>>>>>> page faults. Therefore, when a page fault occurs within a nested 
>>>>>>> domain,
>>>>>>> it apparently should be routed to user space...
>>>>>>>
>>>>>>> If we decide to support page faults on the stage-2 domain in the 
>>>>>>> future,
>>>>>>> we will need to figure out the correct destination of each page fault
>>>>>>> and route it accordingly, either to the parent domain or the user space
>>>>>>> nested domain. Hardware assistance would be beneficial, otherwise the
>>>>>>> software may need to traverse the parent domain, which is not
>>>>>>> performance friendly.
>>>>>>
>>>>>> this is my question. Is it still true the parent domain is always pinned
>>>>>> after the below series? If yes, then it's fine to enable IOPF for nested
>>>>>> domain.
>>>>>>
>>>>>> https://lore.kernel.org/linux-iommu/20241015-jag-iopfv8-v4-0- 
>>>>>> b696ca89ba29@kernel.org/
>>>>>
>>>>> Then, perhaps we could enforce this in iommufd for a short-term purpose?
>>>>>
>>>>> When allocating a hwpt in iommufd, we should enforce that the flags
>>>>> IOMMU_HWPT_ALLOC_NEST_PARENT and IOMMU_HWPT_FAULT_ID_VALID cannot be set
>>>>> at the same time.
>>>>
>>>> Let's consult with other vendors. We need this enforcement because we lack
>>>> a straightforward method to distinguish between PRIs in stage-1 and 
>>>> stage-2
>>>> under nested translation. If other vendors share this requirement, it 
>>>> would
>>>> be appropriate to implement this enforcement in IOMMUFD. Otherwise, we may
>>>> check it in intel iommu driver.
>>>>
>>>> @arm and amd folks. :)
>>>
>>> Yup, SMMUv3 is more or less in the same boat - we *could* reasonably 
>>> manage unpinned S2 for non-PCI devices using the stall model where the 
>>> F_TRANSLATION or F_PERMISSION event tells us all we need, but for ATS 
>>> it's the same thing where by the time the fault has taken a round-trip 
>>> through an ATS response and a PRI request, we've lost the details of 
>>> exactly how it faulted (or if the PRI request was sent eagerly without a 
>>> prior translation request, then we simply have no idea at all). Thus the 
>>> prospective mechanism would be to inject a virtual PRI, wait until we 
>>> see the guest issue a matching CMD_PRI_RESP, then sniff the IPA/GPA for 
>>> the given input address out of the guest pagetables to see if there's 
>>> anything to do at S2 as well/instead. Yuck.
>>
>> Thanks for the response, Robin. It seems we're in the same situation. :(
>> Out of curiosity, will the vPRI response provide the IPA/GPA to the
>> hypervisor? If not, the hypervisor will need to derive it from the GVA on
>> its own, possibly requiring software to traverse the guest page table.
> 
> No, the PRI response command only contains a Page Request Group Index, so 
> the hypervisor would then have to look up all the outstanding requests with 
> that index to retrieve their input addresses and then translate them. The 
> SMMU architecture does technically accommodate a hardware-assisted 
> translation feature (ATOS), but it's optional and rarely implemented - 
> certainly Arm's implementations don't - so in reality that is indeed a 
> going to mean software walks as well.

got it. thanks for the explanation.:)

-- 
Regards,
Yi Liu