Re: [PATCH v7 03/19] iommufd: Replace the hwpt->devices list with iommufd_group

[Baolu Lu <baolu.lu@linux.intel.com>]

On 5/15/23 10:00 PM, Jason Gunthorpe wrote:
> The devices list was used as a simple way to avoid having per-group
> information. Now that this seems to be unavoidable, just commit to
> per-group information fully and remove the devices list from the HWPT.
> 
> The iommufd_group stores the currently assigned HWPT for the entire group
> and we can manage the per-device attach/detach with a list in the
> iommufd_group.

I am preparing the patches to route I/O page faults to user space
through iommufd. The iommufd page fault handler knows the hwpt and the
device pointer, but it needs to convert the device pointer into its
iommufd object id and pass the id to user space.

It's fine that we remove the hwpt->devices here, but perhaps I need to
add the context pointer in ioas later,

struct iommufd_ioas {
         struct io_pagetable iopt;
         struct mutex mutex;
         struct list_head hwpt_list;
+       struct iommufd_ctx *ictx;
  };

and, use below helper to look up the device id.

+u32 iommufd_get_device_id(struct iommufd_ctx *ictx, struct device *dev)
+{
+       struct iommu_group *group = iommu_group_get(dev);
+       u32 dev_id = IOMMUFD_INVALID_OBJ_ID;
+       struct iommufd_group *igroup;
+       struct iommufd_device *cur;
+       unsigned int id;
+
+       if (!group)
+               return IOMMUFD_INVALID_OBJ_ID;
+
+       id = iommu_group_id(group);
+       xa_lock(&ictx->groups);
+       igroup = xa_load(&ictx->groups, id);
+       if (!iommufd_group_try_get(igroup, group)) {
+               xa_unlock(&ictx->groups);
+               iommu_group_put(group);
+               return IOMMUFD_INVALID_OBJ_ID;
+        }
+        xa_unlock(&ictx->groups);
+
+       mutex_lock(&igroup->lock);
+       list_for_each_entry(cur, &igroup->device_list, group_item) {
+               if (cur->dev == dev) {
+                       dev_id = cur->obj.id;
+                       break;
+               }
+       }
+       mutex_unlock(&igroup->lock);
+
+       iommufd_put_group(igroup);
+       iommu_group_put(group);
+
+       return dev_id;
+}

and, use it like this in the fault handler:

        dev_id = iommufd_get_device_id(hwpt->ioas->ictx, dev);
+       if (dev_id == IOMMUFD_INVALID_OBJ_ID)
+               return IOMMU_PAGE_RESP_FAILURE;

Will this look good to you?

> For destruction the flow is organized to make the following patches
> easier, the actual call to iommufd_object_destroy_user() is done at the
> top of the call chain without holding any locks. The HWPT to be destroyed
> is returned out from the locked region to make this possible. Later
> patches create locking that requires this.
> 
> Reviewed-by: Lu Baolu<baolu.lu@linux.intel.com>
> Reviewed-by: Kevin Tian<kevin.tian@intel.com>
> Signed-off-by: Jason Gunthorpe<jgg@nvidia.com>
> ---
>   drivers/iommu/iommufd/device.c          | 100 +++++++++++-------------
>   drivers/iommu/iommufd/hw_pagetable.c    |  22 +-----
>   drivers/iommu/iommufd/iommufd_private.h |  13 ++-
>   3 files changed, 54 insertions(+), 81 deletions(-)

Best regards,
baolu