Re: [PATCH v7 16/16] PCI/TSM: wait for vdevice contexts before removing a DSM

[Aneesh Kumar K.V <aneesh.kumar@kernel.org>]

Jason Gunthorpe <jgg@nvidia.com> writes:

> On Tue, Oct 13, 2026 at 03:14:54PM +0530, Aneesh Kumar K.V wrote:
>> Jason Gunthorpe <jgg@nvidia.com> writes:
>> 
>> > On Mon, Oct 12, 2026 at 03:31:17PM +0530, Aneesh Kumar K.V wrote:
>> >
>> >> With this series, vdevices hold a reference to their parent vIOMMU,
>> >> which pins arm-cca-host. sysfs unbind is disabled, and the SMCCC
>> >> provider device has no hot-unplug path.
>> >
>> > It probably needs some logic like vfio has to prevent unregister from
>> > completing if there are active users. The module pin should prevent it
>> > from ever triggering, but as a safety precaution.
>> >
>> 
>> Are you suggesting that tsm_unregister() should return an error or wait,
>> as device removal does? Returning an error would be much easier to
>> implement by adding an unregistering flag to tsm_dev, set under
>> pci_tsm_rwsem after checking all pf0->context_users counters.
>
> I didn't check it closely but the usual flow is for 'unregister' to
> wait until everything is done using it, possibly with a shootdown
> mechanism.
>
> I'm not sure we can implement a sane shootdown for iommufd's
> consumption of the tsm, but we can at least do the wait.
>

This will require.

@@ -116,6 +119,13 @@ struct tsm_dev {
 	struct device dev;
 	int id;
 	const struct pci_tsm_ops *pci_ops;
+	/* Serializes user acquisition against unregister */
+	struct mutex users_lock;
+	/* Written under users_lock, read with READ_ONCE() by bus code. */
+	bool unregistering;
+	/* Registration reference plus users that must finish before teardown. */
+	refcount_t users;
+	struct completion users_drained;
 };
 

When a vIOMMU pins the tsm_dev, it must also increment users. This
requires a new users_lock, since all existing locking is in pci_tsm.

I can add this as a separate patch, and then we can decide whether to
keep it. I wonder whether we can avoid this complexity, since the vIOMMU
already pins the TSM backend module and we expect unregister to be
called only from module_exit().

-aneesh