PRI support in arm-smmu-v3 driver

PRI support in arm-smmu-v3 driver

[Pavan Kondeti]

Hi

I am trying to understand IO fault handling in Linux w/ SMMUv3. While reading
the code, I understand that SVA domain creation allows taking IO pagefaults.
arm_smmu_enable_iopf() checks if the master support stall upon fault
feature or not. How do we handle page faults for PCIe devices, for which 
transactions cannot safely be stalled? IIUC, The PRI handling in the
driver i.e arm_smmu_priq_thread()->arm_smmu_handle_ppr() is not doing
anything. In the SVA support for SMMUv3 series v7, I see the support for
PRI via "Add support for PRI" patch [1] but it is not merged. 

Can you please clarify if we can support SVA with PCIe devices w/o
pinning the memory?

[1]
https://lore.kernel.org/all/20200519175502.2504091-25-jean-philippe@linaro.org/

Thansk,
Pavan

Re: PRI support in arm-smmu-v3 driver

[Will Deacon]

[+iommu list and usual suspects]

Hi Pavan,

On Tue, Nov 25, 2025 at 02:22:05PM +0530, Pavan Kondeti wrote:
> I am trying to understand IO fault handling in Linux w/ SMMUv3. While reading
> the code, I understand that SVA domain creation allows taking IO pagefaults.
> arm_smmu_enable_iopf() checks if the master support stall upon fault
> feature or not. How do we handle page faults for PCIe devices, for which 
> transactions cannot safely be stalled? IIUC, The PRI handling in the
> driver i.e arm_smmu_priq_thread()->arm_smmu_handle_ppr() is not doing
> anything. In the SVA support for SMMUv3 series v7, I see the support for
> PRI via "Add support for PRI" patch [1] but it is not merged. 
> 
> Can you please clarify if we can support SVA with PCIe devices w/o
> pinning the memory?
> 
> [1]
> https://lore.kernel.org/all/20200519175502.2504091-25-jean-philippe@linaro.org/

The only SVA client we've had for SMMUv3 in the upstream kernel is the
"uacce" thing from HiSilicon which is a platform device (rather than a
PCIe device) and so I think the PRI support just fell by the wayside due
to lack of an upstream user and no ability to test it.

I'm not sure whether or not Jason has plans to implement PRI but maybe
it's something you could help with if you have hardware?

Will

Re: PRI support in arm-smmu-v3 driver

[Jason Gunthorpe]

On Tue, Nov 25, 2025 at 06:00:07PM +0000, Will Deacon wrote:
> [+iommu list and usual suspects]
> 
> Hi Pavan,
> 
> On Tue, Nov 25, 2025 at 02:22:05PM +0530, Pavan Kondeti wrote:
> > I am trying to understand IO fault handling in Linux w/ SMMUv3. While reading
> > the code, I understand that SVA domain creation allows taking IO pagefaults.
> > arm_smmu_enable_iopf() checks if the master support stall upon fault
> > feature or not. How do we handle page faults for PCIe devices, for which 
> > transactions cannot safely be stalled? IIUC, The PRI handling in the
> > driver i.e arm_smmu_priq_thread()->arm_smmu_handle_ppr() is not doing
> > anything. In the SVA support for SMMUv3 series v7, I see the support for
> > PRI via "Add support for PRI" patch [1] but it is not merged. 
> > 
> > Can you please clarify if we can support SVA with PCIe devices w/o
> > pinning the memory?
> > 
> > [1]
> > https://lore.kernel.org/all/20200519175502.2504091-25-jean-philippe@linaro.org/
> 
> The only SVA client we've had for SMMUv3 in the upstream kernel is the
> "uacce" thing from HiSilicon which is a platform device (rather than a
> PCIe device) and so I think the PRI support just fell by the wayside due
> to lack of an upstream user and no ability to test it.

Right, we only support "stall" mode in the driver, not PRI right
now. PRI has a bunch of differences at the SMMU level.

> I'm not sure whether or not Jason has plans to implement PRI but maybe
> it's something you could help with if you have hardware?

I've been waiting for someone who has HW to take this on.

Honestly, I'm not entirely sure what the missing gaps are, at least I
think we need to get the PRI information and package it into the fault
queue and link it back to a PRI response.

Jason

Re: PRI support in arm-smmu-v3 driver

[Jonathan Cameron]

On Tue, 25 Nov 2025 14:03:53 -0400
Jason Gunthorpe <jgg@nvidia.com> wrote:

> On Tue, Nov 25, 2025 at 06:00:07PM +0000, Will Deacon wrote:
> > [+iommu list and usual suspects]
> > 
> > Hi Pavan,
> > 
> > On Tue, Nov 25, 2025 at 02:22:05PM +0530, Pavan Kondeti wrote:  
> > > I am trying to understand IO fault handling in Linux w/ SMMUv3. While reading
> > > the code, I understand that SVA domain creation allows taking IO pagefaults.
> > > arm_smmu_enable_iopf() checks if the master support stall upon fault
> > > feature or not. How do we handle page faults for PCIe devices, for which 
> > > transactions cannot safely be stalled? IIUC, The PRI handling in the
> > > driver i.e arm_smmu_priq_thread()->arm_smmu_handle_ppr() is not doing
> > > anything. In the SVA support for SMMUv3 series v7, I see the support for
> > > PRI via "Add support for PRI" patch [1] but it is not merged. 
> > > 
> > > Can you please clarify if we can support SVA with PCIe devices w/o
> > > pinning the memory?
> > > 
> > > [1]
> > > https://lore.kernel.org/all/20200519175502.2504091-25-jean-philippe@linaro.org/  
> > 
> > The only SVA client we've had for SMMUv3 in the upstream kernel is the
> > "uacce" thing from HiSilicon which is a platform device (rather than a
> > PCIe device) and so I think the PRI support just fell by the wayside due
> > to lack of an upstream user and no ability to test it.  
> 
> Right, we only support "stall" mode in the driver, not PRI right
> now. PRI has a bunch of differences at the SMMU level.
> 
> > I'm not sure whether or not Jason has plans to implement PRI but maybe
> > it's something you could help with if you have hardware?  
> 
> I've been waiting for someone who has HW to take this on.
> 
> Honestly, I'm not entirely sure what the missing gaps are, at least I
> think we need to get the PRI information and package it into the fault
> queue and link it back to a PRI response.

Long ago we did some testing using various 'interesting' PRI supporting
test setups (they weren't what you would call real devices but did let
us exercise the support).

IIRC the above stuff from JPB worked at the time. Given we didn't have
any real endpoints, we didn't push for it to merge.

No idea what status would be today though. It is on our list to come
back to.

Jonathan

> 
> Jason
>