Re: [PATCH v2] PCI/IOV: Add reentrant locking in sriov_add_vfs/sriov_del_vfs for complete serialization

[Dragos Tatulea <dtatulea@nvidia.com>]

On 19.02.26 22:26, Ionut Nechita (Wind River) wrote:
> From: Ionut Nechita <ionut.nechita@windriver.com>
> 
> After reverting commit 05703271c3cd ("PCI/IOV: Add PCI rescan-remove
> locking when enabling/disabling SR-IOV") and moving the lock to
> sriov_numvfs_store(), the path through driver .remove() (e.g. rmmod,
> or manual unbind) that calls pci_disable_sriov() directly remains
> unprotected against concurrent hotplug events. This affects any SR-IOV
> capable driver that calls pci_disable_sriov() from its .remove()
> callback (i40e, ice, mlx5, bnxt, etc.).
> 
> On s390, platform-generated hot-unplug events for VFs can race with
> sriov_del_vfs() when a PF driver is being unloaded. The platform event
> handler takes pci_rescan_remove_lock, but sriov_del_vfs() does not,
> leading to double removal and list corruption.
> 
> We cannot use a plain mutex_lock() here because sriov_del_vfs() may also
> be called from paths that already hold pci_rescan_remove_lock (e.g.
> remove_store -> pci_stop_and_remove_bus_device_locked, or
> sriov_numvfs_store with the lock taken by the previous patch). Using
> mutex_lock() in those cases would deadlock.
> 
> Instead, introduce owner tracking for pci_rescan_remove_lock via a new
> pci_lock_rescan_remove_reentrant() helper. This function checks if the
> current task already holds the lock:
>  - If the lock is not held: acquires it and returns true, providing
>    full serialization against concurrent hotplug events (including
>    platform-generated events on s390).
>  - If the lock is already held by the current task (reentrant call from
>    remove_store or sriov_numvfs_store paths): returns false without
>    re-acquiring, avoiding deadlock while the caller already provides
>    the necessary serialization.
>  - If the lock is held by another task (concurrent hotplug): blocks
>    until the lock is released, then acquires it, providing complete
>    serialization. This is the key improvement over a trylock approach.
> 
> A matching pci_unlock_rescan_remove_reentrant() helper takes the return
> value of the lock function as argument, so callers don't need to
> open-code the conditional unlock.
> 
> The "reentrant" naming is chosen to avoid confusion with existing
> mutex_lock_nested() which is a lockdep annotation concept, not actual
> reentrant locking.
> 
> Note: owner-tracking patterns for reentrant lock behavior exist elsewhere
> in the kernel, for example in the regulator core (drivers/regulator/core.c)
> with rdev->mutex_owner, and in the PPP subsystem (drivers/net/ppp/
> ppp_generic.c) with xmit_recursion->owner.
> 
> The declarations are placed in include/linux/pci.h alongside the existing
> pci_lock_rescan_remove()/pci_unlock_rescan_remove() declarations to
> maintain API consistency and allow use by external drivers if needed.
> 
> Fixes: 18f9e9d150fc ("PCI/IOV: Factor out sriov_add_vfs()")
> Cc: stable@vger.kernel.org
> Signed-off-by: Ionut Nechita <ionut_n2001@yahoo.com>
> Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
Thanks! According to [1]:

Tested-by: Dragos Tatulea <dtatulea@nvidia.com>

[1] https://lore.kernel.org/linux-pci/a02222aa-64a2-43b9-86f3-a31b4668206c@nvidia.com/

Thanks,
Dragos