[PATCH] drm/xe/pf: Keep VF LMEM BAR size low if no VFs enabled

[Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>]

When VFs are enabled on dGFX the driver resizes the PF VF_LMEM_BAR to
fit the requested layout. After VFs are disabled the PF VF BAR
size is left as-is. On platforms with tight MMIO apertures a
subsequent unplug/rescan followed by another enable may fail with:

  "VF BAR …: can't assign; no space"

because the PCI core reserves address space based on the (now large) VF
template, often multiplied by totalvfs.

v2:
 - Rename helper to restore_vf_vram_bar_size() (Michal)
 - Use xe->sriov.pf.device_total_vfs instead of pci_sriov_get_totalvfs(),
   which may be capped (Michal)
 - Switch logging to %pe (Michal)
 - Call restore unconditionally on enable-fail path (Michal)
   (drop vf_vram_bar_resized flag)

Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/5937
Fixes: 94eae6ee4c2d ("drm/xe/pf: Set VF LMEM BAR size")
Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
Cc: Michał Wajdeczko <michal.wajdeczko@intel.com>
Cc: Michał Winiarski <michal.winiarski@intel.com>
---
 drivers/gpu/drm/xe/xe_pci_sriov.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/drivers/gpu/drm/xe/xe_pci_sriov.c b/drivers/gpu/drm/xe/xe_pci_sriov.c
index af05db07162e..4caa82aad140 100644
--- a/drivers/gpu/drm/xe/xe_pci_sriov.c
+++ b/drivers/gpu/drm/xe/xe_pci_sriov.c
@@ -144,6 +144,19 @@ static int resize_vf_vram_bar(struct xe_device *xe, int num_vfs)
 	return pci_iov_vf_bar_set_size(pdev, VF_LMEM_BAR, __fls(sizes));
 }
 
+static void restore_vf_vram_bar_size(struct xe_device *xe)
+{
+	int err;
+
+	if (!IS_DGFX(xe))
+		return;
+
+	err = resize_vf_vram_bar(xe, xe->sriov.pf.device_total_vfs);
+	if (err)
+		xe_sriov_info(xe, "Failed to restore VF LMEM BAR size: %pe\n",
+			      ERR_PTR(err));
+}
+
 static int pf_enable_vfs(struct xe_device *xe, int num_vfs)
 {
 	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
@@ -194,6 +207,7 @@ static int pf_enable_vfs(struct xe_device *xe, int num_vfs)
 	return num_vfs;
 
 failed:
+	restore_vf_vram_bar_size(xe);
 	pf_unprovision_vfs(xe, num_vfs);
 	xe_pm_runtime_put(xe);
 out:
@@ -218,6 +232,8 @@ static int pf_disable_vfs(struct xe_device *xe)
 
 	pci_disable_sriov(pdev);
 
+	restore_vf_vram_bar_size(xe);
+
 	pf_reset_vfs(xe, num_vfs);
 
 	pf_unprovision_vfs(xe, num_vfs);
-- 
2.31.1