[RFC PATCH 03/19] vfio/pci: Add PCI recovery access guards

[Shameer Kolothum <skolothumtho@nvidia.com>]

Add a pair of helpers to wrap each operation which touches the device.
access_begin() takes recovery_lock for reading and refuses if the device
is not open, or if recovery is blocking access. The callers come in later
patches.

access_end() drops the lock without looking at the recovery state, so
only call it after access_begin() returned 0. On failure the lock is
already gone. Both helpers key off pci_recovery_supported, which is
fixed for the lifetime of the device, so the pair stays balanced.

The lock is taken even when userspace has not enabled recovery. Enabling
takes recovery_lock for writing, which waits for anything already in
flight. Without that, an operation which started before enable could
still be touching the device when the first error arrives, and there would
be nothing to wait on.

access_blocked is checked either way. Nothing sets it yet, so nothing
which works today gets rejected.

The cost is one rwsem acquire per guarded access on devices that never
turn recovery on. For BAR traffic that is once per width-sized access,
alongside the memory_lock read already taken there.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_priv.h |  3 +++
 drivers/vfio/pci/vfio_pci_core.c | 21 +++++++++++++++++++++
 2 files changed, 24 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index 4e7162234a2e..6daf51669d05 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -73,6 +73,9 @@ u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_core_device *vdev);
 void vfio_pci_memory_unlock_and_restore(struct vfio_pci_core_device *vdev,
 					u16 cmd);
 
+int vfio_pci_core_access_begin(struct vfio_pci_core_device *vdev);
+void vfio_pci_core_access_end(struct vfio_pci_core_device *vdev);
+
 #ifdef CONFIG_VFIO_PCI_IGD
 bool vfio_pci_is_intel_display(struct pci_dev *pdev);
 int vfio_pci_igd_init(struct vfio_pci_core_device *vdev);
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 8de586e4bb73..4194d44d6530 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1747,6 +1747,27 @@ static ssize_t vfio_pci_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 	return ret;
 }
 
+int vfio_pci_core_access_begin(struct vfio_pci_core_device *vdev)
+{
+	if (!vdev->pci_recovery_supported)
+		return 0;
+
+	down_read(&vdev->recovery_lock);
+	if (unlikely(!vdev->pci_recovery_device_open ||
+		     vdev->pci_recovery_access_blocked)) {
+		up_read(&vdev->recovery_lock);
+		return -EIO;
+	}
+
+	return 0;
+}
+
+void vfio_pci_core_access_end(struct vfio_pci_core_device *vdev)
+{
+	if (vdev->pci_recovery_supported)
+		up_read(&vdev->recovery_lock);
+}
+
 ssize_t vfio_pci_core_read(struct vfio_device *core_vdev, char __user *buf,
 		size_t count, loff_t *ppos)
 {
-- 
2.43.0