Re: [PATCH v6 01/10] iommufd/viommu: Add IOMMUFD_OBJ_VDEVICE and IOMMU_VDEVICE_ALLOC ioctl

[Nicolin Chen <nicolinc@nvidia.com>]

On Thu, Oct 31, 2024 at 10:29:41AM -0300, Jason Gunthorpe wrote:
> On Wed, Oct 30, 2024 at 02:35:27PM -0700, Nicolin Chen wrote:
> > +void iommufd_vdevice_destroy(struct iommufd_object *obj)
> > +{
> > +	struct iommufd_vdevice *vdev =
> > +		container_of(obj, struct iommufd_vdevice, obj);
> > +	struct iommufd_viommu *viommu = vdev->viommu;
> > +
> > +	/* xa_cmpxchg is okay to fail if alloc returned -EEXIST previously */
> > +	xa_cmpxchg(&viommu->vdevs, vdev->id, vdev, NULL, GFP_KERNEL);
> 
> There are crazy races that would cause this not to work. Another
> thread could have successfully destroyed whatever caused EEXIST and
> the successfully registered this same vdev to the same id. Then this
> will wrongly erase the other threads entry.
>
> It would be better to skip the erase directly if the EEXIST unwind is
> being taken.

Hmm, is the "another thread" an alloc() or a destroy()? It doesn't
seem to me that there could be another destroy() on the same object
since this current destroy() is the abort to an unfinalized object.
And it doesn't seem that another alloc() will get the same vdev ptr
since every vdev allocation in the alloc() will be different?

That being said, I think we could play safer with the followings:
-------------------------------------------------------------------
@@ -88,7 +88,6 @@ void iommufd_vdevice_destroy(struct iommufd_object *obj)
                container_of(obj, struct iommufd_vdevice, obj);
        struct iommufd_viommu *viommu = vdev->viommu;

-       /* xa_cmpxchg is okay to fail if alloc returned -EEXIST previously */
        xa_cmpxchg(&viommu->vdevs, vdev->id, vdev, NULL, GFP_KERNEL);
        refcount_dec(&viommu->obj.users);
        put_device(vdev->dev);
@@ -128,18 +127,19 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd)
                goto out_put_idev;
        }

+       curr = xa_cmpxchg(&viommu->vdevs, virt_id, NULL, vdev, GFP_KERNEL);
+       if (curr) {
+               iommufd_object_abort(ucmd->ictx, &vdev->obj);
+               rc = xa_err(curr) ?: -EEXIST;
+               goto out_put_idev;
+       }
+
        vdev->id = virt_id;
        vdev->dev = idev->dev;
        get_device(idev->dev);
        vdev->viommu = viommu;
        refcount_inc(&viommu->obj.users);

-       curr = xa_cmpxchg(&viommu->vdevs, virt_id, NULL, vdev, GFP_KERNEL);
-       if (curr) {
-               rc = xa_err(curr) ?: -EEXIST;
-               goto out_abort;
-       }
-
        cmd->out_vdevice_id = vdev->obj.id;
        rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd));
        if (rc)
-------------------------------------------------------------------

Thanks
Nicolin