[PATCH] iommufd: modify iommufd_fault_iopf_enable limitation

[PATCH] iommufd: modify iommufd_fault_iopf_enable limitation

[Zhangfei Gao]

iommufd_fault_iopf_enable has limitation to PRI on PCI/SRIOV VFs
because the PRI might be a shared resource and current iommu
subsystem is not ready to support enabling/disabling PRI on a VF
without any impact on others.

However, we have devices that appear as PCI but are actually on the
AMBA bus. These fake PCI devices have PASID capability, support
stall as well as SRIOV, so remove the limitation for these devices.

Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/iommufd/fault.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/iommufd/fault.c b/drivers/iommu/iommufd/fault.c
index bca956d496bd..8b3e34250dae 100644
--- a/drivers/iommu/iommufd/fault.c
+++ b/drivers/iommu/iommufd/fault.c
@@ -10,6 +10,7 @@
 #include <linux/module.h>
 #include <linux/mutex.h>
 #include <linux/pci.h>
+#include <linux/pci-ats.h>
 #include <linux/poll.h>
 #include <uapi/linux/iommufd.h>
 
@@ -27,8 +28,12 @@ static int iommufd_fault_iopf_enable(struct iommufd_device *idev)
 	 * resource between PF and VFs. There is no coordination for this
 	 * shared capability. This waits for a vPRI reset to recover.
 	 */
-	if (dev_is_pci(dev) && to_pci_dev(dev)->is_virtfn)
-		return -EINVAL;
+	if (dev_is_pci(dev)) {
+		struct pci_dev *pdev = to_pci_dev(dev);
+
+		if (pdev->is_virtfn && pci_pri_supported(pdev))
+			return -EINVAL;
+	}
 
 	mutex_lock(&idev->iopf_lock);
 	/* Device iopf has already been on. */
-- 
2.25.1

Re: [PATCH] iommufd: modify iommufd_fault_iopf_enable limitation

[Zhangfei Gao]

On Mon, 28 Oct 2024 at 11:32, Zhangfei Gao <zhangfei.gao@linaro.org> wrote:
>
> iommufd_fault_iopf_enable has limitation to PRI on PCI/SRIOV VFs
> because the PRI might be a shared resource and current iommu
> subsystem is not ready to support enabling/disabling PRI on a VF
> without any impact on others.
>
> However, we have devices that appear as PCI but are actually on the
> AMBA bus. These fake PCI devices have PASID capability, support
> stall as well as SRIOV, so remove the limitation for these devices.
>
> Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> ---
>  drivers/iommu/iommufd/fault.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/iommu/iommufd/fault.c b/drivers/iommu/iommufd/fault.c
> index bca956d496bd..8b3e34250dae 100644
> --- a/drivers/iommu/iommufd/fault.c
> +++ b/drivers/iommu/iommufd/fault.c
> @@ -10,6 +10,7 @@
>  #include <linux/module.h>
>  #include <linux/mutex.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ats.h>
>  #include <linux/poll.h>
>  #include <uapi/linux/iommufd.h>
>
> @@ -27,8 +28,12 @@ static int iommufd_fault_iopf_enable(struct iommufd_device *idev)
>          * resource between PF and VFs. There is no coordination for this
>          * shared capability. This waits for a vPRI reset to recover.
>          */
> -       if (dev_is_pci(dev) && to_pci_dev(dev)->is_virtfn)
> -               return -EINVAL;
> +       if (dev_is_pci(dev)) {
> +               struct pci_dev *pdev = to_pci_dev(dev);
> +
> +               if (pdev->is_virtfn && pci_pri_supported(pdev))
> +                       return -EINVAL;
> +       }
>
>         mutex_lock(&idev->iopf_lock);
>         /* Device iopf has already been on. */
> --
> 2.25.1
>

Hi, Jason

Would you mind also taking a look at this.

Thanks

Re: [PATCH] iommufd: modify iommufd_fault_iopf_enable limitation

[Jason Gunthorpe]

On Wed, Nov 06, 2024 at 05:47:09AM +0000, Zhangfei Gao wrote:
> On Mon, 28 Oct 2024 at 11:32, Zhangfei Gao <zhangfei.gao@linaro.org> wrote:
> >
> > iommufd_fault_iopf_enable has limitation to PRI on PCI/SRIOV VFs
> > because the PRI might be a shared resource and current iommu
> > subsystem is not ready to support enabling/disabling PRI on a VF
> > without any impact on others.
> >
> > However, we have devices that appear as PCI but are actually on the
> > AMBA bus. These fake PCI devices have PASID capability, support
> > stall as well as SRIOV, so remove the limitation for these devices.
> >
> > Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
> > Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> > ---
> >  drivers/iommu/iommufd/fault.c | 9 +++++++--
> >  1 file changed, 7 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/iommu/iommufd/fault.c b/drivers/iommu/iommufd/fault.c
> > index bca956d496bd..8b3e34250dae 100644
> > --- a/drivers/iommu/iommufd/fault.c
> > +++ b/drivers/iommu/iommufd/fault.c
> > @@ -10,6 +10,7 @@
> >  #include <linux/module.h>
> >  #include <linux/mutex.h>
> >  #include <linux/pci.h>
> > +#include <linux/pci-ats.h>
> >  #include <linux/poll.h>
> >  #include <uapi/linux/iommufd.h>
> >
> > @@ -27,8 +28,12 @@ static int iommufd_fault_iopf_enable(struct iommufd_device *idev)
> >          * resource between PF and VFs. There is no coordination for this
> >          * shared capability. This waits for a vPRI reset to recover.
> >          */
> > -       if (dev_is_pci(dev) && to_pci_dev(dev)->is_virtfn)
> > -               return -EINVAL;
> > +       if (dev_is_pci(dev)) {
> > +               struct pci_dev *pdev = to_pci_dev(dev);
> > +
> > +               if (pdev->is_virtfn && pci_pri_supported(pdev))
> > +                       return -EINVAL;
> > +       }
> >
> >         mutex_lock(&idev->iopf_lock);
> >         /* Device iopf has already been on. */
> >
> 
> Hi, Jason
> 
> Would you mind also taking a look at this.

Lu? Are you OK with this?

Jason

Re: [PATCH] iommufd: modify iommufd_fault_iopf_enable limitation

[Baolu Lu]

On 11/6/24 21:59, Jason Gunthorpe wrote:
> On Wed, Nov 06, 2024 at 05:47:09AM +0000, Zhangfei Gao wrote:
>> On Mon, 28 Oct 2024 at 11:32, Zhangfei Gao<zhangfei.gao@linaro.org> wrote:
>>> iommufd_fault_iopf_enable has limitation to PRI on PCI/SRIOV VFs
>>> because the PRI might be a shared resource and current iommu
>>> subsystem is not ready to support enabling/disabling PRI on a VF
>>> without any impact on others.
>>>
>>> However, we have devices that appear as PCI but are actually on the
>>> AMBA bus. These fake PCI devices have PASID capability, support
>>> stall as well as SRIOV, so remove the limitation for these devices.
>>>
>>> Signed-off-by: Zhangfei Gao<zhangfei.gao@linaro.org>
>>> Signed-off-by: Lu Baolu<baolu.lu@linux.intel.com>
>>> ---
>>>   drivers/iommu/iommufd/fault.c | 9 +++++++--
>>>   1 file changed, 7 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/iommu/iommufd/fault.c b/drivers/iommu/iommufd/fault.c
>>> index bca956d496bd..8b3e34250dae 100644
>>> --- a/drivers/iommu/iommufd/fault.c
>>> +++ b/drivers/iommu/iommufd/fault.c
>>> @@ -10,6 +10,7 @@
>>>   #include <linux/module.h>
>>>   #include <linux/mutex.h>
>>>   #include <linux/pci.h>
>>> +#include <linux/pci-ats.h>
>>>   #include <linux/poll.h>
>>>   #include <uapi/linux/iommufd.h>
>>>
>>> @@ -27,8 +28,12 @@ static int iommufd_fault_iopf_enable(struct iommufd_device *idev)
>>>           * resource between PF and VFs. There is no coordination for this
>>>           * shared capability. This waits for a vPRI reset to recover.
>>>           */
>>> -       if (dev_is_pci(dev) && to_pci_dev(dev)->is_virtfn)
>>> -               return -EINVAL;
>>> +       if (dev_is_pci(dev)) {
>>> +               struct pci_dev *pdev = to_pci_dev(dev);
>>> +
>>> +               if (pdev->is_virtfn && pci_pri_supported(pdev))
>>> +                       return -EINVAL;
>>> +       }
>>>
>>>          mutex_lock(&idev->iopf_lock);
>>>          /* Device iopf has already been on. */
>>>
>> Hi, Jason
>>
>> Would you mind also taking a look at this.
> Lu? Are you OK with this?

This change looks good to me. But the s-o-b chain would make more sense
if we can make it like this,

Co-developed-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Zhangfei Gao<zhangfei.gao@linaro.org>

With this addressed,

Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>

--
baolu

Re: [PATCH] iommufd: modify iommufd_fault_iopf_enable limitation

[Zhangfei Gao]

On Thu, 7 Nov 2024 at 01:52, Baolu Lu <baolu.lu@linux.intel.com> wrote:
>
> On 11/6/24 21:59, Jason Gunthorpe wrote:
> > On Wed, Nov 06, 2024 at 05:47:09AM +0000, Zhangfei Gao wrote:
> >> On Mon, 28 Oct 2024 at 11:32, Zhangfei Gao<zhangfei.gao@linaro.org> wrote:
> >>> iommufd_fault_iopf_enable has limitation to PRI on PCI/SRIOV VFs
> >>> because the PRI might be a shared resource and current iommu
> >>> subsystem is not ready to support enabling/disabling PRI on a VF
> >>> without any impact on others.
> >>>
> >>> However, we have devices that appear as PCI but are actually on the
> >>> AMBA bus. These fake PCI devices have PASID capability, support
> >>> stall as well as SRIOV, so remove the limitation for these devices.
> >>>
> >>> Signed-off-by: Zhangfei Gao<zhangfei.gao@linaro.org>
> >>> Signed-off-by: Lu Baolu<baolu.lu@linux.intel.com>
> >>> ---
> >>>   drivers/iommu/iommufd/fault.c | 9 +++++++--
> >>>   1 file changed, 7 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/drivers/iommu/iommufd/fault.c b/drivers/iommu/iommufd/fault.c
> >>> index bca956d496bd..8b3e34250dae 100644
> >>> --- a/drivers/iommu/iommufd/fault.c
> >>> +++ b/drivers/iommu/iommufd/fault.c
> >>> @@ -10,6 +10,7 @@
> >>>   #include <linux/module.h>
> >>>   #include <linux/mutex.h>
> >>>   #include <linux/pci.h>
> >>> +#include <linux/pci-ats.h>
> >>>   #include <linux/poll.h>
> >>>   #include <uapi/linux/iommufd.h>
> >>>
> >>> @@ -27,8 +28,12 @@ static int iommufd_fault_iopf_enable(struct iommufd_device *idev)
> >>>           * resource between PF and VFs. There is no coordination for this
> >>>           * shared capability. This waits for a vPRI reset to recover.
> >>>           */
> >>> -       if (dev_is_pci(dev) && to_pci_dev(dev)->is_virtfn)
> >>> -               return -EINVAL;
> >>> +       if (dev_is_pci(dev)) {
> >>> +               struct pci_dev *pdev = to_pci_dev(dev);
> >>> +
> >>> +               if (pdev->is_virtfn && pci_pri_supported(pdev))
> >>> +                       return -EINVAL;
> >>> +       }
> >>>
> >>>          mutex_lock(&idev->iopf_lock);
> >>>          /* Device iopf has already been on. */
> >>>
> >> Hi, Jason
> >>
> >> Would you mind also taking a look at this.
> > Lu? Are you OK with this?
>
> This change looks good to me. But the s-o-b chain would make more sense
> if we can make it like this,
>
> Co-developed-by: Lu Baolu <baolu.lu@linux.intel.com>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> Signed-off-by: Zhangfei Gao<zhangfei.gao@linaro.org>
>
> With this addressed,
>
> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>

Thanks Baolu,
Have fixed it.

Thanks

[PATCH v2] iommufd: modify iommufd_fault_iopf_enable limitation

[Zhangfei Gao]

iommufd_fault_iopf_enable has limitation to PRI on PCI/SRIOV VFs
because the PRI might be a shared resource and current iommu
subsystem is not ready to support enabling/disabling PRI on a VF
without any impact on others.

However, we have devices that appear as PCI but are actually on the
AMBA bus. These fake PCI devices have PASID capability, support
stall as well as SRIOV, so remove the limitation for these devices.

Co-developed-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/iommufd/fault.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/iommufd/fault.c b/drivers/iommu/iommufd/fault.c
index 80efef1c127d..c5b0aa719948 100644
--- a/drivers/iommu/iommufd/fault.c
+++ b/drivers/iommu/iommufd/fault.c
@@ -10,6 +10,7 @@
 #include <linux/module.h>
 #include <linux/mutex.h>
 #include <linux/pci.h>
+#include <linux/pci-ats.h>
 #include <linux/poll.h>
 #include <uapi/linux/iommufd.h>
 
@@ -27,8 +28,12 @@ static int iommufd_fault_iopf_enable(struct iommufd_device *idev)
 	 * resource between PF and VFs. There is no coordination for this
 	 * shared capability. This waits for a vPRI reset to recover.
 	 */
-	if (dev_is_pci(dev) && to_pci_dev(dev)->is_virtfn)
-		return -EINVAL;
+	if (dev_is_pci(dev)) {
+		struct pci_dev *pdev = to_pci_dev(dev);
+
+		if (pdev->is_virtfn && pci_pri_supported(pdev))
+			return -EINVAL;
+	}
 
 	mutex_lock(&idev->iopf_lock);
 	/* Device iopf has already been on. */
-- 
2.25.1

Re: [PATCH v2] iommufd: modify iommufd_fault_iopf_enable limitation

[Jason Gunthorpe]

On Thu, Nov 07, 2024 at 04:37:11AM +0000, Zhangfei Gao wrote:
> iommufd_fault_iopf_enable has limitation to PRI on PCI/SRIOV VFs
> because the PRI might be a shared resource and current iommu
> subsystem is not ready to support enabling/disabling PRI on a VF
> without any impact on others.
> 
> However, we have devices that appear as PCI but are actually on the
> AMBA bus. These fake PCI devices have PASID capability, support
> stall as well as SRIOV, so remove the limitation for these devices.
> 
> Co-developed-by: Lu Baolu <baolu.lu@linux.intel.com>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
> ---
>  drivers/iommu/iommufd/fault.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)

Applied to iommufd thanks

Jason