[PATCH v2 0/7] gpu: nova-core: boot GSP with vGPU enabled

[PATCH v2 0/7] gpu: nova-core: boot GSP with vGPU enabled

[Zhi Wang]

Booting up GSP with vGPU enabled is part of the first milestone (M1)
together with the Rust fwctl abstraction [1] and the nova-core fwctl
driver [2] for upstream vGPU support. It allows us to validate the basic
GSP boot flow with vGPU enabled and upload vGPU types even before the
remaining nova-core dependencies are ready. The nova-vgpu WIP patches
for all milestones can be found at [3].

This version is based on drm-rust-next plus Alexandre's GSP boot process
consolidation series [4].

v2:

- Rebase on top of Alexandre's GSP boot process consolidation series.
- Drop the FSP response, FSP documentation, and GspBootContext patches
  that are already in drm-rust-next or superseded by the prerequisite
  boot consolidation series.
- Change pci_sriov_get_totalvfs() to return u16 and update existing C
  callers accordingly.
- Make the Rust sriov_get_totalvfs() helper return u16 directly.
- Rework the FSP PRC vGPU mode query to use typed subcommand, object ID,
  flags, request, and response structures.
- Move vGPU state detection before GSP boot into a read-only VgpuManager,
  avoiding Mutex/Cell based mutation during boot.
- Add a HAL method for the vGPU capability gate.
- Split the SetRegistry changes into a dynamic-entry refactor and the
  RMSetSriovMode functional change.
- Rework WPR2 heap sizing to consume VgpuManager, keep the vGPU heap-size
  helper in gsp/fw.rs, and drop the 1VM heap-size special case.

[1] https://lore.kernel.org/rust-for-linux/20260217204909.211793-1-zhiw@nvidia.com/
[2] https://lore.kernel.org/rust-for-linux/20260305190936.398590-1-zhiw@nvidia.com/
[3] https://github.com/zhiwang-nvidia/nova-core/tree/zhi/nova-vgpu-wip
[4] https://lore.kernel.org/all/20260622-nova-bootcontext-v2-13-0ddeafc06f5d@nvidia.com/

Zhi Wang (7):
  PCI/IOV: Return u16 from pci_sriov_get_totalvfs()
  rust: pci: Add sriov_get_totalvfs() helper
  gpu: nova-core: read vGPU mode from FSP via PRC protocol
  gpu: nova-core: add vGPU preludes
  gpu: nova-core: build SetRegistry entries dynamically
  gpu: nova-core: set RMSetSriovMode when vGPU is enabled
  gpu: nova-core: reserve larger WPR2 heap for vGPU

 drivers/crypto/hisilicon/qm.c                 |   8 +-
 .../crypto/intel/qat/qat_common/adf_sriov.c   |   6 +-
 drivers/gpu/drm/xe/xe_sriov_pf.c              |   6 +-
 drivers/gpu/nova-core/fb.rs                   |  25 ++-
 drivers/gpu/nova-core/fsp.rs                  | 194 +++++++++++++++++-
 drivers/gpu/nova-core/gpu.rs                  |  12 ++
 drivers/gpu/nova-core/gpu/hal.rs              |   3 +
 drivers/gpu/nova-core/gpu/hal/gh100.rs        |  12 +-
 drivers/gpu/nova-core/gpu/hal/tu102.rs        |   5 +
 drivers/gpu/nova-core/gsp.rs                  |   3 +
 drivers/gpu/nova-core/gsp/boot.rs             |  11 +-
 drivers/gpu/nova-core/gsp/commands.rs         |  79 ++++---
 drivers/gpu/nova-core/gsp/fw.rs               |  10 +
 drivers/gpu/nova-core/mctp.rs                 |   3 +
 drivers/gpu/nova-core/nova_core.rs            |   1 +
 drivers/gpu/nova-core/vgpu.rs                 |  60 ++++++
 drivers/misc/genwqe/card_base.c               |   6 +-
 .../net/ethernet/cavium/thunder/nic_main.c    |   2 +-
 drivers/net/ethernet/emulex/benet/be_main.c   |   3 +-
 .../net/ethernet/mellanox/mlx5/core/sriov.c   |   3 +-
 drivers/net/ethernet/sfc/ef10_sriov.c         |   2 +-
 drivers/pci/iov.c                             |   2 +-
 drivers/vdpa/octeon_ep/octep_vdpa_main.c      |   4 +-
 include/linux/pci.h                           |   4 +-
 rust/kernel/pci.rs                            |   6 +
 25 files changed, 401 insertions(+), 69 deletions(-)
 create mode 100644 drivers/gpu/nova-core/vgpu.rs

-- 
2.51.0

[PATCH v2 1/7] PCI/IOV: Return u16 from pci_sriov_get_totalvfs()

[Zhi Wang]

pci_sriov_get_totalvfs() reports a VF count, not an errno-style
status. It returns 0 when SR-IOV is unavailable or the device is not a
PF, and otherwise returns the PF's driver_max_VFs value.

driver_max_VFs is stored as a u16 in struct pci_sriov. It is derived
from the SR-IOV TotalVFs field or from a driver-provided limit, so the
implementation cannot return a negative value.

Change the declaration, CONFIG_PCI_IOV stub, and implementation to
return u16. Update callers to store the result in u16 variables, remove
obsolete negative-value checks, and use unsigned format specifiers where
needed.

Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: linux-pci@vger.kernel.org
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
 drivers/crypto/hisilicon/qm.c                   | 8 +++++---
 drivers/crypto/intel/qat/qat_common/adf_sriov.c | 6 +++---
 drivers/gpu/drm/xe/xe_sriov_pf.c                | 6 ++----
 drivers/misc/genwqe/card_base.c                 | 6 ++----
 drivers/net/ethernet/cavium/thunder/nic_main.c  | 2 +-
 drivers/net/ethernet/emulex/benet/be_main.c     | 3 ++-
 drivers/net/ethernet/mellanox/mlx5/core/sriov.c | 3 ++-
 drivers/net/ethernet/sfc/ef10_sriov.c           | 2 +-
 drivers/pci/iov.c                               | 2 +-
 drivers/vdpa/octeon_ep/octep_vdpa_main.c        | 4 ++--
 include/linux/pci.h                             | 4 ++--
 11 files changed, 23 insertions(+), 23 deletions(-)

diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c
index 3ca47e2a9719..13b45fcb582b 100644
--- a/drivers/crypto/hisilicon/qm.c
+++ b/drivers/crypto/hisilicon/qm.c
@@ -3878,7 +3878,8 @@ static int qm_func_shaper_enable(struct hisi_qm *qm, u32 fun_index, u32 qos)
 	struct device *dev = &qm->pdev->dev;
 	struct qm_shaper_factor t_factor;
 	u32 ir = qos * QM_QOS_RATE;
-	int ret, total_vfs, i;
+	u16 total_vfs;
+	int ret, i;
 
 	total_vfs = pci_sriov_get_totalvfs(qm->pdev);
 	if (fun_index > total_vfs)
@@ -4188,7 +4189,8 @@ static void hisi_qm_init_vf_qos(struct hisi_qm *qm, int total_func)
 int hisi_qm_sriov_enable(struct pci_dev *pdev, int max_vfs)
 {
 	struct hisi_qm *qm = pci_get_drvdata(pdev);
-	int pre_existing_vfs, num_vfs, total_vfs, ret;
+	int pre_existing_vfs, num_vfs, ret;
+	u16 total_vfs;
 
 	ret = qm_pm_get_sync(qm);
 	if (ret)
@@ -4203,7 +4205,7 @@ int hisi_qm_sriov_enable(struct pci_dev *pdev, int max_vfs)
 	}
 
 	if (max_vfs > total_vfs) {
-		pci_err(pdev, "%d VFs is more than total VFs %d!\n", max_vfs, total_vfs);
+		pci_err(pdev, "%d VFs is more than total VFs %u!\n", max_vfs, total_vfs);
 		ret = -ERANGE;
 		goto err_put_sync;
 	}
diff --git a/drivers/crypto/intel/qat/qat_common/adf_sriov.c b/drivers/crypto/intel/qat/qat_common/adf_sriov.c
index 8bf0fe1fcb4d..317b80c1772f 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_sriov.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_sriov.c
@@ -52,7 +52,7 @@ void adf_schedule_vf2pf_handler(struct adf_accel_vf_info *vf_info)
 static int adf_enable_sriov(struct adf_accel_dev *accel_dev)
 {
 	struct pci_dev *pdev = accel_to_pci_dev(accel_dev);
-	int totalvfs = pci_sriov_get_totalvfs(pdev);
+	u16 totalvfs = pci_sriov_get_totalvfs(pdev);
 	struct adf_hw_device_data *hw_data = accel_dev->hw_device;
 	struct adf_accel_vf_info *vf_info;
 	int i;
@@ -148,7 +148,7 @@ static int adf_do_disable_sriov(struct adf_accel_dev *accel_dev)
 static int adf_do_enable_sriov(struct adf_accel_dev *accel_dev)
 {
 	struct pci_dev *pdev = accel_to_pci_dev(accel_dev);
-	int totalvfs = pci_sriov_get_totalvfs(pdev);
+	u16 totalvfs = pci_sriov_get_totalvfs(pdev);
 	unsigned long val;
 	int ret;
 
@@ -237,7 +237,7 @@ void adf_reenable_sriov(struct adf_accel_dev *accel_dev)
 void adf_disable_sriov(struct adf_accel_dev *accel_dev)
 {
 	struct adf_hw_device_data *hw_data = accel_dev->hw_device;
-	int totalvfs = pci_sriov_get_totalvfs(accel_to_pci_dev(accel_dev));
+	u16 totalvfs = pci_sriov_get_totalvfs(accel_to_pci_dev(accel_dev));
 	struct adf_accel_vf_info *vf;
 	int i;
 
diff --git a/drivers/gpu/drm/xe/xe_sriov_pf.c b/drivers/gpu/drm/xe/xe_sriov_pf.c
index 33bd754d138f..1a1c4978e5f3 100644
--- a/drivers/gpu/drm/xe/xe_sriov_pf.c
+++ b/drivers/gpu/drm/xe/xe_sriov_pf.c
@@ -58,10 +58,8 @@ bool xe_sriov_pf_readiness(struct xe_device *xe)
 {
 	struct device *dev = xe->drm.dev;
 	struct pci_dev *pdev = to_pci_dev(dev);
-	int totalvfs = pci_sriov_get_totalvfs(pdev);
-	int newlimit = min_t(u16, wanted_max_vfs(xe), totalvfs);
-
-	xe_assert(xe, totalvfs <= U16_MAX);
+	u16 totalvfs = pci_sriov_get_totalvfs(pdev);
+	u16 newlimit = min_t(u16, wanted_max_vfs(xe), totalvfs);
 
 	if (!dev_is_pf(dev))
 		return false;
diff --git a/drivers/misc/genwqe/card_base.c b/drivers/misc/genwqe/card_base.c
index 86bfa82723ff..259d2f9c9416 100644
--- a/drivers/misc/genwqe/card_base.c
+++ b/drivers/misc/genwqe/card_base.c
@@ -360,10 +360,10 @@ static bool genwqe_setup_vf_jtimer(struct genwqe_dev *cd)
 	unsigned int vf;
 	u32 T = genwqe_T_psec(cd);
 	u64 x;
-	int totalvfs;
+	u16 totalvfs;
 
 	totalvfs = pci_sriov_get_totalvfs(pci_dev);
-	if (totalvfs <= 0)
+	if (!totalvfs)
 		return false;
 
 	for (vf = 0; vf < totalvfs; vf++) {
@@ -1132,8 +1132,6 @@ static int genwqe_pci_setup(struct genwqe_dev *cd)
 	}
 
 	cd->num_vfs = pci_sriov_get_totalvfs(pci_dev);
-	if (cd->num_vfs < 0)
-		cd->num_vfs = 0;
 
 	err = genwqe_read_ids(cd);
 	if (err)
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c b/drivers/net/ethernet/cavium/thunder/nic_main.c
index 0ec65ec634df..33faa40381d3 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -98,7 +98,7 @@ static u64 nic_reg_read(struct nicpf *nic, u64 offset)
 /* PF -> VF mailbox communication APIs */
 static void nic_enable_mbx_intr(struct nicpf *nic)
 {
-	int vf_cnt = pci_sriov_get_totalvfs(nic->pdev);
+	u16 vf_cnt = pci_sriov_get_totalvfs(nic->pdev);
 
 #define INTR_MASK(vfs) ((vfs < 64) ? (BIT_ULL(vfs) - 1) : (~0ull))
 
diff --git a/drivers/net/ethernet/emulex/benet/be_main.c b/drivers/net/ethernet/emulex/benet/be_main.c
index ed302f5ec476..21cc5113ab92 100644
--- a/drivers/net/ethernet/emulex/benet/be_main.c
+++ b/drivers/net/ethernet/emulex/benet/be_main.c
@@ -4441,7 +4441,8 @@ static void be_calculate_pf_pool_rss_tables(struct be_adapter *adapter)
 static int be_get_sriov_config(struct be_adapter *adapter)
 {
 	struct be_resources res = {0};
-	int max_vfs, old_vfs;
+	u16 max_vfs;
+	int old_vfs;
 
 	be_cmd_get_profile_config(adapter, &res, NULL, ACTIVE_PROFILE_TYPE,
 				  RESOURCE_LIMITS, 0);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sriov.c b/drivers/net/ethernet/mellanox/mlx5/core/sriov.c
index bf6f631cf2ce..7087f1a4d364 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/sriov.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/sriov.c
@@ -298,7 +298,8 @@ int mlx5_sriov_init(struct mlx5_core_dev *dev)
 {
 	struct mlx5_core_sriov *sriov = &dev->priv.sriov;
 	struct pci_dev *pdev = dev->pdev;
-	int total_vfs, i;
+	u16 total_vfs;
+	int i;
 
 	if (!mlx5_core_is_pf(dev))
 		return 0;
diff --git a/drivers/net/ethernet/sfc/ef10_sriov.c b/drivers/net/ethernet/sfc/ef10_sriov.c
index f98f1707d1a9..5b339b055761 100644
--- a/drivers/net/ethernet/sfc/ef10_sriov.c
+++ b/drivers/net/ethernet/sfc/ef10_sriov.c
@@ -263,7 +263,7 @@ int efx_ef10_vswitching_probe_pf(struct efx_nic *efx)
 	struct net_device *net_dev = efx->net_dev;
 	int rc;
 
-	if (pci_sriov_get_totalvfs(efx->pci_dev) <= 0) {
+	if (!pci_sriov_get_totalvfs(efx->pci_dev)) {
 		/* vswitch not needed as we have no VFs */
 		efx_ef10_vadaptor_alloc_set_features(efx);
 		return 0;
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index 91ac4e37ecb9..db9828bfff52 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -1277,7 +1277,7 @@ EXPORT_SYMBOL_GPL(pci_sriov_set_totalvfs);
  * SRIOV capability value of TotalVFs or the value of driver_max_VFs
  * if the driver reduced it.  Otherwise 0.
  */
-int pci_sriov_get_totalvfs(struct pci_dev *dev)
+u16 pci_sriov_get_totalvfs(struct pci_dev *dev)
 {
 	if (!dev->is_physfn)
 		return 0;
diff --git a/drivers/vdpa/octeon_ep/octep_vdpa_main.c b/drivers/vdpa/octeon_ep/octep_vdpa_main.c
index 31a02e7fd7f2..4b032b757ee9 100644
--- a/drivers/vdpa/octeon_ep/octep_vdpa_main.c
+++ b/drivers/vdpa/octeon_ep/octep_vdpa_main.c
@@ -796,13 +796,13 @@ static int octep_vdpa_pf_setup(struct octep_pf *octpf)
 {
 	u8 __iomem *addr = octpf->base[OCTEP_HW_MBOX_BAR];
 	struct pci_dev *pdev = octpf->pdev;
-	int totalvfs;
+	u16 totalvfs;
 	size_t len;
 	u64 val;
 
 	totalvfs = pci_sriov_get_totalvfs(pdev);
 	if (unlikely(!totalvfs)) {
-		dev_info(&pdev->dev, "Total VFs are %d in PF sriov configuration\n", totalvfs);
+		dev_info(&pdev->dev, "Total VFs are %u in PF sriov configuration\n", totalvfs);
 		return 0;
 	}
 
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 2c4454583c11..1994a88afbc2 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -2538,7 +2538,7 @@ void pci_iov_remove_virtfn(struct pci_dev *dev, int id);
 int pci_num_vf(struct pci_dev *dev);
 int pci_vfs_assigned(struct pci_dev *dev);
 int pci_sriov_set_totalvfs(struct pci_dev *dev, u16 numvfs);
-int pci_sriov_get_totalvfs(struct pci_dev *dev);
+u16 pci_sriov_get_totalvfs(struct pci_dev *dev);
 int pci_sriov_configure_simple(struct pci_dev *dev, int nr_virtfn);
 resource_size_t pci_iov_resource_size(struct pci_dev *dev, int resno);
 int pci_iov_vf_bar_set_size(struct pci_dev *dev, int resno, int size);
@@ -2590,7 +2590,7 @@ static inline int pci_vfs_assigned(struct pci_dev *dev)
 { return 0; }
 static inline int pci_sriov_set_totalvfs(struct pci_dev *dev, u16 numvfs)
 { return 0; }
-static inline int pci_sriov_get_totalvfs(struct pci_dev *dev)
+static inline u16 pci_sriov_get_totalvfs(struct pci_dev *dev)
 { return 0; }
 #define pci_sriov_configure_simple	NULL
 static inline resource_size_t pci_iov_resource_size(struct pci_dev *dev, int resno)
-- 
2.51.0

[PATCH v2 2/7] rust: pci: Add sriov_get_totalvfs() helper

[Zhi Wang]

Expose pci_sriov_get_totalvfs() to Rust PCI drivers so they can query
how many SR-IOV VFs a device supports.

The C helper now returns u16 and reports 0 when SR-IOV is unavailable
or the device is not a PF, so mirror that API directly instead of
modeling 0 VFs as an error.

Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: linux-pci@vger.kernel.org
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
 rust/kernel/pci.rs | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index 5071cae6543f..adcb99fd0a2e 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -450,6 +450,12 @@ pub fn pci_class(&self) -> Class {
         // SAFETY: `self.as_raw` is a valid pointer to a `struct pci_dev`.
         Class::from_raw(unsafe { (*self.as_raw()).class })
     }
+
+    /// Returns the total number of VFs, or 0 if SR-IOV is not available.
+    pub fn sriov_get_totalvfs(&self) -> u16 {
+        // SAFETY: `self.as_raw()` is a valid pointer to a `struct pci_dev`.
+        unsafe { bindings::pci_sriov_get_totalvfs(self.as_raw()) }
+    }
 }
 
 impl<'a> Device<device::Core<'a>> {
-- 
2.51.0

[PATCH v2 3/7] gpu: nova-core: read vGPU mode from FSP via PRC protocol

[Zhi Wang]

Add support for querying the vGPU mode configuration from FSP using
the PRC (Product Reconfiguration Control) protocol. PRC is an API
system exposed through FSP's Management Partition that allows querying
device configuration "knobs" without firmware updates.

Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
 drivers/gpu/nova-core/fsp.rs  | 195 ++++++++++++++++++++++++++++++++--
 drivers/gpu/nova-core/mctp.rs |   3 +
 2 files changed, 190 insertions(+), 8 deletions(-)

diff --git a/drivers/gpu/nova-core/fsp.rs b/drivers/gpu/nova-core/fsp.rs
index 4b97d1fb505e..2bf01c2d1175 100644
--- a/drivers/gpu/nova-core/fsp.rs
+++ b/drivers/gpu/nova-core/fsp.rs
@@ -51,6 +51,56 @@
 
 mod hal;
 
+/// PRC message sub-command.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+#[repr(u8)]
+enum PrcMessageSubcmd {
+    /// Read a PRC knob value.
+    Read = 0x0c,
+}
+
+impl From<PrcMessageSubcmd> for u8 {
+    fn from(value: PrcMessageSubcmd) -> Self {
+        value as u8
+    }
+}
+
+/// PRC object identifier.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+#[repr(u8)]
+enum PrcObjectId {
+    /// vGPU mode configuration knob.
+    VgpuMode = 0x29,
+}
+
+impl From<PrcObjectId> for u8 {
+    fn from(value: PrcObjectId) -> Self {
+        value as u8
+    }
+}
+
+kernel::impl_flags!(
+    /// PRC request flags.
+    #[derive(Clone, Copy, Default, PartialEq, Eq)]
+    struct PrcFlags(u8);
+
+    /// Individual PRC request flag.
+    #[derive(Clone, Copy, PartialEq, Eq)]
+    enum PrcFlag {
+        /// Request the active knob value for the current boot.
+        Active = 1 << 1,
+    }
+);
+
+/// vGPU operating mode as reported by FSP via the PRC protocol.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum VgpuMode {
+    /// vGPU support is disabled on this GPU.
+    Disabled,
+    /// vGPU support is enabled on this GPU.
+    Enabled,
+}
+
 /// FSP command response payload (`NVDM_PAYLOAD_COMMAND_RESPONSE`).
 #[repr(C, packed)]
 #[derive(Clone, Copy)]
@@ -60,6 +110,62 @@ struct NvdmPayloadCommandResponse {
     error_code: u32,
 }
 
+/// PRC message payload.
+///
+/// Sent to FSP to query or modify a device configuration knob.
+#[repr(C, packed)]
+#[derive(Clone, Copy)]
+struct NvdmPayloadPrc {
+    sub_message_id: u8,
+    flags: u8,
+    object_id: u8,
+    reserved: u8,
+}
+
+impl NvdmPayloadPrc {
+    /// Constructs a PRC payload from typed protocol fields.
+    fn new(subcmd: PrcMessageSubcmd, object_id: PrcObjectId, flags: PrcFlags) -> Self {
+        Self {
+            sub_message_id: subcmd.into(),
+            flags: flags.into(),
+            object_id: object_id.into(),
+            reserved: 0,
+        }
+    }
+}
+
+// SAFETY: NvdmPayloadPrc is a packed C struct with only integral fields.
+unsafe impl AsBytes for NvdmPayloadPrc {}
+
+/// PRC response payload containing the knob state value.
+#[repr(C, packed)]
+#[derive(Clone, Copy)]
+struct NvdmPayloadPrcResponse {
+    value_low: u8,
+    value_high: u8,
+    reserved1: u8,
+    reserved2: u8,
+}
+
+impl NvdmPayloadPrcResponse {
+    /// Returns the PRC knob value as a little-endian 16-bit integer.
+    fn value(self) -> u16 {
+        u16::from(self.value_low) | (u16::from(self.value_high) << 8)
+    }
+}
+
+impl TryFrom<NvdmPayloadPrcResponse> for VgpuMode {
+    type Error = kernel::error::Error;
+
+    fn try_from(value: NvdmPayloadPrcResponse) -> Result<Self> {
+        match value.value() {
+            0 => Ok(VgpuMode::Disabled),
+            1 => Ok(VgpuMode::Enabled),
+            _ => Err(EINVAL),
+        }
+    }
+}
+
 /// Common MCTP and NVDM headers shared by all FSP messages.
 #[repr(C, packed)]
 #[derive(Clone, Copy)]
@@ -84,16 +190,27 @@ fn new(nvdm_type: NvdmType) -> Self {
     }
 }
 
-/// Complete FSP response structure with MCTP and NVDM headers.
+/// Common FSP response header with MCTP, NVDM and command response payloads.
 #[repr(C, packed)]
 #[derive(Clone, Copy)]
-struct FspResponse {
+struct FspResponseHeader {
     header: FspMessageHeader,
     response: NvdmPayloadCommandResponse,
 }
 
-// SAFETY: FspResponse is a packed C struct with only integral fields.
-unsafe impl FromBytes for FspResponse {}
+// SAFETY: FspResponseHeader is a packed C struct with only integral fields.
+unsafe impl FromBytes for FspResponseHeader {}
+
+/// Complete FSP PRC response including the knob state payload.
+#[repr(C, packed)]
+#[derive(Clone, Copy)]
+struct FspPrcResponse {
+    header: FspResponseHeader,
+    prc_data: NvdmPayloadPrcResponse,
+}
+
+// SAFETY: FspPrcResponse is a packed C struct with only integral fields.
+unsafe impl FromBytes for FspPrcResponse {}
 
 /// Trait implemented by types representing a message to send to FSP.
 ///
@@ -182,10 +299,35 @@ fn new<'a>(
 // bytes are initialized.
 unsafe impl AsBytes for FspCotMessage {}
 
+/// Complete FSP PRC message.
+#[repr(C, packed)]
+#[derive(Clone, Copy)]
+struct FspPrcMessage {
+    header: FspMessageHeader,
+    prc: NvdmPayloadPrc,
+}
+
+impl FspPrcMessage {
+    /// Constructs a PRC message.
+    fn new(subcmd: PrcMessageSubcmd, object_id: PrcObjectId, flags: PrcFlags) -> Self {
+        Self {
+            header: FspMessageHeader::new(NvdmType::Prc),
+            prc: NvdmPayloadPrc::new(subcmd, object_id, flags),
+        }
+    }
+}
+
+// SAFETY: FspPrcMessage is a packed C struct with only integral fields.
+unsafe impl AsBytes for FspPrcMessage {}
+
 impl MessageToFsp for FspCotMessage {
     const NVDM_TYPE: NvdmType = NvdmType::Cot;
 }
 
+impl MessageToFsp for FspPrcMessage {
+    const NVDM_TYPE: NvdmType = NvdmType::Prc;
+}
+
 /// Bundled arguments for FMC boot via FSP Chain of Trust.
 pub(crate) struct FmcBootArgs {
     chipset: Chipset,
@@ -272,10 +414,11 @@ fn send_sync_fsp<M>(&mut self, dev: &device::Device, bar: Bar0<'_>, msg: &M) ->
             dev_err!(dev, "FSP response error: {:?}\n", e);
         })?;
 
-        let (response, _) = FspResponse::from_bytes_prefix(&response_buf[..]).ok_or_else(|| {
-            dev_err!(dev, "FSP response too small: {}\n", response_buf.len());
-            EIO
-        })?;
+        let (response, _) =
+            FspResponseHeader::from_bytes_prefix(&response_buf[..]).ok_or_else(|| {
+                dev_err!(dev, "FSP response too small: {}\n", response_buf.len());
+                EIO
+            })?;
 
         let mctp_header = response.header.mctp_header;
         let nvdm_header = response.header.nvdm_header;
@@ -323,6 +466,42 @@ fn send_sync_fsp<M>(&mut self, dev: &device::Device, bar: Bar0<'_>, msg: &M) ->
         Ok(response_buf)
     }
 
+    /// Reads the active vGPU mode from FSP using the PRC protocol.
+    ///
+    /// Queries FSP's Management Partition for the active vGPU mode knob value.
+    #[expect(dead_code)]
+    pub(crate) fn read_vgpu_mode(
+        &mut self,
+        dev: &device::Device<device::Bound>,
+        bar: Bar0<'_>,
+    ) -> Result<VgpuMode> {
+        let msg = KBox::new(
+            FspPrcMessage::new(
+                PrcMessageSubcmd::Read,
+                PrcObjectId::VgpuMode,
+                PrcFlags::from(PrcFlag::Active),
+            ),
+            GFP_KERNEL,
+        )?;
+
+        let response_buf = self.send_sync_fsp(dev, bar, &*msg)?;
+        let (prc_response, _) =
+            FspPrcResponse::from_bytes_prefix(&response_buf[..]).ok_or_else(|| {
+                dev_err!(dev, "PRC response too small: {}\n", response_buf.len());
+                EIO
+            })?;
+
+        let prc_data = prc_response.prc_data;
+
+        VgpuMode::try_from(prc_data).inspect_err(|_| {
+            dev_err!(
+                dev,
+                "Unexpected vGPU mode value: {:#x}\n",
+                prc_data.value()
+            );
+        })
+    }
+
     /// Boots GSP FMC via FSP Chain of Trust.
     ///
     /// Builds the CoT message from the pre-configured [`FmcBootArgs`], sends it
diff --git a/drivers/gpu/nova-core/mctp.rs b/drivers/gpu/nova-core/mctp.rs
index 482786e07bc7..b203c632bf20 100644
--- a/drivers/gpu/nova-core/mctp.rs
+++ b/drivers/gpu/nova-core/mctp.rs
@@ -13,6 +13,8 @@
 #[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
 #[repr(u8)]
 pub(crate) enum NvdmType {
+    /// PRC (Product Reconfiguration Control) message.
+    Prc = 0x13,
     #[default]
     /// Chain of Trust boot message.
     Cot = 0x14,
@@ -25,6 +27,7 @@ impl TryFrom<u8> for NvdmType {
 
     fn try_from(value: u8) -> Result<Self, Self::Error> {
         match value {
+            x if x == u8::from(Self::Prc) => Ok(Self::Prc),
             x if x == u8::from(Self::Cot) => Ok(Self::Cot),
             x if x == u8::from(Self::FspResponse) => Ok(Self::FspResponse),
             _ => Err(value),
-- 
2.51.0

[PATCH v2 4/7] gpu: nova-core: add vGPU preludes

[Zhi Wang]

Detect vGPU state before GSP boot so later boot steps can consume a
stable view of whether vGPU is enabled and how many SR-IOV VFs are
available.

Introduce VgpuManager to keep the detected vGPU state. The manager uses
a GPU HAL capability gate, pci_sriov_get_totalvfs(), and the FSP PRC
vGPU mode knob to decide whether vGPU is enabled for the current boot.
vGPU is considered enabled only when at least two VFs are available.
The state is read-only after construction and is referenced from
GspBootContext instead of being copied into it.

Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
 drivers/gpu/nova-core/fsp.rs           |  1 -
 drivers/gpu/nova-core/gpu.rs           | 12 ++++++
 drivers/gpu/nova-core/gpu/hal.rs       |  3 ++
 drivers/gpu/nova-core/gpu/hal/gh100.rs | 12 +++++-
 drivers/gpu/nova-core/gpu/hal/tu102.rs |  5 +++
 drivers/gpu/nova-core/gsp.rs           |  2 +
 drivers/gpu/nova-core/gsp/boot.rs      |  7 +++
 drivers/gpu/nova-core/nova_core.rs     |  1 +
 drivers/gpu/nova-core/vgpu.rs          | 60 ++++++++++++++++++++++++++
 9 files changed, 101 insertions(+), 2 deletions(-)
 create mode 100644 drivers/gpu/nova-core/vgpu.rs

diff --git a/drivers/gpu/nova-core/fsp.rs b/drivers/gpu/nova-core/fsp.rs
index 2bf01c2d1175..3b5006750693 100644
--- a/drivers/gpu/nova-core/fsp.rs
+++ b/drivers/gpu/nova-core/fsp.rs
@@ -469,7 +469,6 @@ fn send_sync_fsp<M>(&mut self, dev: &device::Device, bar: Bar0<'_>, msg: &M) ->
     /// Reads the active vGPU mode from FSP using the PRC protocol.
     ///
     /// Queries FSP's Management Partition for the active vGPU mode knob value.
-    #[expect(dead_code)]
     pub(crate) fn read_vgpu_mode(
         &mut self,
         dev: &device::Device<device::Bound>,
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index e5ebd79c9020..646eb6c63c21 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -30,6 +30,7 @@
         GspBootMethod, //
     },
     regs,
+    vgpu::VgpuManager,
 };
 
 mod hal;
@@ -138,6 +139,11 @@ pub(crate) const fn arch(self) -> Architecture {
     pub(crate) fn pci_config_mirror_range(self) -> Range<u32> {
         hal::gpu_hal(self).pci_config_mirror_range()
     }
+
+    /// Returns whether this chipset can support vGPU.
+    pub(crate) fn supports_vgpu(self) -> bool {
+        hal::gpu_hal(self).supports_vgpu(self)
+    }
 }
 
 // TODO
@@ -267,6 +273,8 @@ struct GspResources<'gpu> {
     // TODO: use different resource types for each boot method, and make the relevant Gsp methods
     // generic against them.
     fsp: Option<Fsp>,
+    /// vGPU state detected before GSP boot.
+    vgpu: VgpuManager,
     /// GSP runtime data.
     #[pin]
     gsp: Gsp,
@@ -311,6 +319,7 @@ fn drop(self: Pin<&mut Self>) {
                     gsp_falcon: &*this.gsp_falcon,
                     sec2_falcon: &*this.sec2_falcon,
                     fsp: this.fsp.as_mut(),
+                    vgpu: &*this.vgpu,
                 },
                 bundle,
             )
@@ -366,6 +375,8 @@ pub(crate) fn new(
                     GspBootMethod::Fsp => Some(Fsp::wait_secure_boot(dev, bar, spec.chipset)?),
                 },
 
+                vgpu: VgpuManager::new(pdev, spec.chipset, bar, fsp.as_mut())?,
+
                 gsp <- Gsp::new(pdev),
 
                 // This member must be initialized last, so the `UnloadBundle` can never be dropped
@@ -378,6 +389,7 @@ pub(crate) fn new(
                     gsp_falcon,
                     sec2_falcon,
                     fsp: fsp.as_mut(),
+                    vgpu,
                 })?,
             }),
 
diff --git a/drivers/gpu/nova-core/gpu/hal.rs b/drivers/gpu/nova-core/gpu/hal.rs
index 3f25882d0e56..2116c71242ec 100644
--- a/drivers/gpu/nova-core/gpu/hal.rs
+++ b/drivers/gpu/nova-core/gpu/hal.rs
@@ -27,6 +27,9 @@ pub(crate) trait GpuHal {
 
     /// Returns the address range of the PCI config mirror space.
     fn pci_config_mirror_range(&self) -> Range<u32>;
+
+    /// Returns whether this chipset can support vGPU.
+    fn supports_vgpu(&self, chipset: Chipset) -> bool;
 }
 
 pub(super) fn gpu_hal(chipset: Chipset) -> &'static dyn GpuHal {
diff --git a/drivers/gpu/nova-core/gpu/hal/gh100.rs b/drivers/gpu/nova-core/gpu/hal/gh100.rs
index e3f8ba0fab33..8e18206961ae 100644
--- a/drivers/gpu/nova-core/gpu/hal/gh100.rs
+++ b/drivers/gpu/nova-core/gpu/hal/gh100.rs
@@ -7,7 +7,13 @@
     prelude::*, //
 };
 
-use crate::driver::Bar0;
+use crate::{
+    driver::Bar0,
+    gpu::{
+        Architecture,
+        Chipset, //
+    },
+};
 
 use super::GpuHal;
 
@@ -28,6 +34,10 @@ fn pci_config_mirror_range(&self) -> Range<u32> {
 
         PCI_CONFIG_MIRROR_START..PCI_CONFIG_MIRROR_START + PCI_CONFIG_MIRROR_SIZE
     }
+
+    fn supports_vgpu(&self, chipset: Chipset) -> bool {
+        matches!(chipset.arch(), Architecture::BlackwellGB20x)
+    }
 }
 
 const GH100: Gh100 = Gh100;
diff --git a/drivers/gpu/nova-core/gpu/hal/tu102.rs b/drivers/gpu/nova-core/gpu/hal/tu102.rs
index b0732e53edea..1e2c7bbdb4ad 100644
--- a/drivers/gpu/nova-core/gpu/hal/tu102.rs
+++ b/drivers/gpu/nova-core/gpu/hal/tu102.rs
@@ -32,6 +32,7 @@
 
 use crate::{
     driver::Bar0,
+    gpu::Chipset,
     regs, //
 };
 
@@ -94,6 +95,10 @@ fn pci_config_mirror_range(&self) -> Range<u32> {
 
         PCI_CONFIG_MIRROR_START..PCI_CONFIG_MIRROR_START + PCI_CONFIG_MIRROR_SIZE
     }
+
+    fn supports_vgpu(&self, _chipset: Chipset) -> bool {
+        false
+    }
 }
 
 const TU102: Tu102 = Tu102;
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index ff438506070a..6821008d48d9 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -51,6 +51,7 @@
         },
     },
     num,
+    vgpu::VgpuManager,
 };
 
 pub(crate) const GSP_PAGE_SHIFT: usize = 12;
@@ -64,6 +65,7 @@ pub(crate) struct GspBootContext<'a> {
     pub(crate) gsp_falcon: &'a Falcon<GspFalcon>,
     pub(crate) sec2_falcon: &'a Falcon<Sec2Falcon>,
     pub(crate) fsp: Option<&'a mut Fsp>,
+    pub(crate) vgpu: &'a VgpuManager,
 }
 
 impl<'a> GspBootContext<'a> {
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 1e0d4793e96d..0a33cf4dd975 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -47,6 +47,13 @@ pub(crate) fn boot(
         let dev = pdev.as_ref();
         let hal = super::hal::gsp_hal(chipset);
 
+        dev_dbg!(
+            dev,
+            "vGPU enabled: {}, total VFs: {}\n",
+            ctx.vgpu.enabled(),
+            ctx.vgpu.total_vfs()
+        );
+
         let gsp_fw = KBox::pin_init(GspFirmware::new(dev, chipset, FIRMWARE_VERSION), GFP_KERNEL)?;
 
         let fb_layout = FbLayout::new(chipset, bar, &gsp_fw)?;
diff --git a/drivers/gpu/nova-core/nova_core.rs b/drivers/gpu/nova-core/nova_core.rs
index 735b8e17c6b6..2df2f773ec8e 100644
--- a/drivers/gpu/nova-core/nova_core.rs
+++ b/drivers/gpu/nova-core/nova_core.rs
@@ -26,6 +26,7 @@
 mod regs;
 mod sbuffer;
 mod vbios;
+mod vgpu;
 
 pub(crate) const MODULE_NAME: &core::ffi::CStr = <LocalModule as kernel::ModuleMetadata>::NAME;
 
diff --git a/drivers/gpu/nova-core/vgpu.rs b/drivers/gpu/nova-core/vgpu.rs
new file mode 100644
index 000000000000..08fa37d80b28
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu.rs
@@ -0,0 +1,60 @@
+// SPDX-License-Identifier: GPL-2.0
+
+use kernel::{
+    device,
+    pci,
+    prelude::*, //
+};
+
+use crate::{
+    driver::Bar0,
+    fsp::{
+        Fsp,
+        VgpuMode, //
+    },
+    gpu::Chipset,
+};
+
+/// vGPU state detected during GPU construction.
+pub(crate) struct VgpuManager {
+    enabled: bool,
+    total_vfs: u16,
+}
+
+impl VgpuManager {
+    /// Creates a vGPU manager by querying SR-IOV and the FSP PRC vGPU knob.
+    pub(crate) fn new(
+        pdev: &pci::Device<device::Bound>,
+        chipset: Chipset,
+        bar: Bar0<'_>,
+        fsp: Option<&mut Fsp>,
+    ) -> Result<Self> {
+        let total_vfs = if chipset.supports_vgpu() {
+            pdev.sriov_get_totalvfs()
+        } else {
+            0
+        };
+
+        let enabled = if total_vfs < 2 {
+            false
+        } else if let Some(fsp) = fsp {
+            let mode = fsp.read_vgpu_mode(pdev.as_ref(), bar)?;
+
+            mode == VgpuMode::Enabled
+        } else {
+            false
+        };
+
+        Ok(Self { enabled, total_vfs })
+    }
+
+    /// Returns whether vGPU mode is enabled for this boot.
+    pub(crate) fn enabled(&self) -> bool {
+        self.enabled
+    }
+
+    /// Returns the total number of SR-IOV VFs supported by this device.
+    pub(crate) fn total_vfs(&self) -> u16 {
+        self.total_vfs
+    }
+}
-- 
2.51.0

[PATCH v2 5/7] gpu: nova-core: build SetRegistry entries dynamically

[Zhi Wang]

Convert SetRegistry to store registry entries in a KVec instead of a
fixed-size array. This keeps the existing registry contents unchanged
while making the command ready for entries that are appended based on
runtime state.

No functional change is intended.

Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
 drivers/gpu/nova-core/gsp/boot.rs     |  2 +-
 drivers/gpu/nova-core/gsp/commands.rs | 73 +++++++++++++++------------
 2 files changed, 41 insertions(+), 34 deletions(-)

diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 0a33cf4dd975..55ea8e38aa79 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -94,7 +94,7 @@ pub(crate) fn boot(
             self.cmdq
                 .send_command_no_wait(bar, commands::SetSystemInfo::new(pdev, chipset))?;
             self.cmdq
-                .send_command_no_wait(bar, commands::SetRegistry::new())?;
+                .send_command_no_wait(bar, commands::SetRegistry::new()?)?;
 
             hal.post_boot(&self, &mut ctx, &gsp_fw)?;
 
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index f84de9f4f045..3dccb012a37e 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -66,37 +66,44 @@ struct RegistryEntry {
 
 /// The `SetRegistry` command.
 pub(crate) struct SetRegistry {
-    entries: [RegistryEntry; Self::NUM_ENTRIES],
+    entries: KVec<RegistryEntry>,
 }
 
 impl SetRegistry {
-    // For now we hard-code the registry entries. Future work will allow others to
-    // be added as module parameters.
-    const NUM_ENTRIES: usize = 3;
-
     /// Creates a new `SetRegistry` command, using a set of hardcoded entries.
-    pub(crate) fn new() -> Self {
-        Self {
-            entries: [
-                // RMSecBusResetEnable - enables PCI secondary bus reset
-                RegistryEntry {
-                    key: "RMSecBusResetEnable",
-                    value: 1,
-                },
-                // RMForcePcieConfigSave - forces GSP-RM to preserve PCI configuration registers on
-                // any PCI reset.
-                RegistryEntry {
-                    key: "RMForcePcieConfigSave",
-                    value: 1,
-                },
-                // RMDevidCheckIgnore - allows GSP-RM to boot even if the PCI dev ID is not found
-                // in the internal product name database.
-                RegistryEntry {
-                    key: "RMDevidCheckIgnore",
-                    value: 1,
-                },
-            ],
-        }
+    pub(crate) fn new() -> Result<Self> {
+        let mut entries = KVec::new();
+
+        // RMSecBusResetEnable - enables PCI secondary bus reset.
+        entries.push(
+            RegistryEntry {
+                key: "RMSecBusResetEnable",
+                value: 1,
+            },
+            GFP_KERNEL,
+        )?;
+
+        // RMForcePcieConfigSave - forces GSP-RM to preserve PCI configuration registers on any PCI
+        // reset.
+        entries.push(
+            RegistryEntry {
+                key: "RMForcePcieConfigSave",
+                value: 1,
+            },
+            GFP_KERNEL,
+        )?;
+
+        // RMDevidCheckIgnore - allows GSP-RM to boot even if the PCI dev ID is not found in the
+        // internal product name database.
+        entries.push(
+            RegistryEntry {
+                key: "RMDevidCheckIgnore",
+                value: 1,
+            },
+            GFP_KERNEL,
+        )?;
+
+        Ok(Self { entries })
     }
 }
 
@@ -107,15 +114,15 @@ impl CommandToGsp for SetRegistry {
     type InitError = Infallible;
 
     fn init(&self) -> impl Init<Self::Command, Self::InitError> {
-        Self::Command::init(Self::NUM_ENTRIES as u32, self.variable_payload_len() as u32)
+        Self::Command::init(self.entries.len() as u32, self.variable_payload_len() as u32)
     }
 
     fn variable_payload_len(&self) -> usize {
         let mut key_size = 0;
-        for i in 0..Self::NUM_ENTRIES {
-            key_size += self.entries[i].key.len() + 1; // +1 for NULL terminator
+        for entry in self.entries.iter() {
+            key_size += entry.key.len() + 1; // +1 for NULL terminator
         }
-        Self::NUM_ENTRIES * size_of::<fw::commands::PackedRegistryEntry>() + key_size
+        self.entries.len() * size_of::<fw::commands::PackedRegistryEntry>() + key_size
     }
 
     fn init_variable_payload(
@@ -123,12 +130,12 @@ fn init_variable_payload(
         dst: &mut SBufferIter<core::array::IntoIter<&mut [u8], 2>>,
     ) -> Result {
         let string_data_start_offset = size_of::<Self::Command>()
-            + Self::NUM_ENTRIES * size_of::<fw::commands::PackedRegistryEntry>();
+            + self.entries.len() * size_of::<fw::commands::PackedRegistryEntry>();
 
         // Array for string data.
         let mut string_data = KVec::new();
 
-        for entry in self.entries.iter().take(Self::NUM_ENTRIES) {
+        for entry in self.entries.iter() {
             dst.write_all(
                 fw::commands::PackedRegistryEntry::new(
                     (string_data_start_offset + string_data.len()) as u32,
-- 
2.51.0

[PATCH v2 6/7] gpu: nova-core: set RMSetSriovMode when vGPU is enabled

[Zhi Wang]

The registry object "RMSetSriovMode" is required to be set when vGPU is
enabled.

Set "RMSetSriovMode" to 1 when nova-core is loading the GSP firmware and
initializing the GSP registry objects, if vGPU is enabled.

Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
 drivers/gpu/nova-core/gsp/boot.rs     |  2 +-
 drivers/gpu/nova-core/gsp/commands.rs | 16 ++++++++++++++--
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 55ea8e38aa79..c607081e8242 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -94,7 +94,7 @@ pub(crate) fn boot(
             self.cmdq
                 .send_command_no_wait(bar, commands::SetSystemInfo::new(pdev, chipset))?;
             self.cmdq
-                .send_command_no_wait(bar, commands::SetRegistry::new()?)?;
+                .send_command_no_wait(bar, commands::SetRegistry::new(ctx.vgpu)?)?;
 
             hal.post_boot(&self, &mut ctx, &gsp_fw)?;
 
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 3dccb012a37e..bf2b569f5578 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -33,6 +33,7 @@
         },
     },
     sbuffer::SBufferIter,
+    vgpu::VgpuManager,
 };
 
 /// The `GspSetSystemInfo` command.
@@ -70,8 +71,8 @@ pub(crate) struct SetRegistry {
 }
 
 impl SetRegistry {
-    /// Creates a new `SetRegistry` command, using a set of hardcoded entries.
-    pub(crate) fn new() -> Result<Self> {
+    /// Creates a new `SetRegistry` command.
+    pub(crate) fn new(vgpu: &VgpuManager) -> Result<Self> {
         let mut entries = KVec::new();
 
         // RMSecBusResetEnable - enables PCI secondary bus reset.
@@ -103,6 +104,17 @@ pub(crate) fn new() -> Result<Self> {
             GFP_KERNEL,
         )?;
 
+        if vgpu.enabled() {
+            // RMSetSriovMode - required when vGPU is enabled.
+            entries.push(
+                RegistryEntry {
+                    key: "RMSetSriovMode",
+                    value: 1,
+                },
+                GFP_KERNEL,
+            )?;
+        }
+
         Ok(Self { entries })
     }
 }
-- 
2.51.0

[PATCH v2 7/7] gpu: nova-core: reserve larger WPR2 heap for vGPU

[Zhi Wang]

From: Zhi Wang <zhi.wang.ai@sent.com>

Reserve a larger GSP WPR2 heap when vGPU is enabled, using the VF count
reported by VgpuManager to select the heap size and to populate the VF
partition count in the GSP WPR metadata.

Use the baremetal heap sizing path when vGPU is disabled. For vGPU,
reserve 581 MiB for up to 32 VFs and 1370 MiB for larger VF counts.

Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
 drivers/gpu/nova-core/fb.rs       | 25 ++++++++++++++++++++-----
 drivers/gpu/nova-core/gsp.rs      |  1 +
 drivers/gpu/nova-core/gsp/boot.rs |  2 +-
 drivers/gpu/nova-core/gsp/fw.rs   | 10 ++++++++++
 4 files changed, 32 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/nova-core/fb.rs b/drivers/gpu/nova-core/fb.rs
index 725e428154cf..b5b1f6c13edc 100644
--- a/drivers/gpu/nova-core/fb.rs
+++ b/drivers/gpu/nova-core/fb.rs
@@ -24,7 +24,8 @@
     gpu::Chipset,
     gsp,
     num::FromSafeCast,
-    regs, //
+    regs,
+    vgpu::VgpuManager, //
 };
 
 mod hal;
@@ -171,7 +172,12 @@ pub(crate) struct FbLayout {
 
 impl FbLayout {
     /// Computes the FB layout for `chipset` required to run the `gsp_fw` GSP firmware.
-    pub(crate) fn new(chipset: Chipset, bar: Bar0<'_>, gsp_fw: &GspFirmware) -> Result<Self> {
+    pub(crate) fn new(
+        chipset: Chipset,
+        bar: Bar0<'_>,
+        gsp_fw: &GspFirmware,
+        vgpu: &VgpuManager,
+    ) -> Result<Self> {
         let hal = hal::fb_hal(chipset);
 
         let fb = {
@@ -234,10 +240,19 @@ pub(crate) fn new(chipset: Chipset, bar: Bar0<'_>, gsp_fw: &GspFirmware) -> Resu
             FbRange(elf_addr..elf_addr + elf_size)
         };
 
+        let vf_partition_count = if vgpu.enabled() {
+            vgpu.total_vfs().try_into().map_err(|_| EINVAL)?
+        } else {
+            0
+        };
+
         let wpr2_heap = {
             const WPR2_HEAP_DOWN_ALIGN: Alignment = Alignment::new::<SZ_1M>();
-            let wpr2_heap_size =
-                gsp::LibosParams::from_chipset(chipset).wpr_heap_size(chipset, fb.end)?;
+            let wpr2_heap_size = if vgpu.enabled() {
+                gsp::vgpu_fw_heap_size(u32::from(vgpu.total_vfs()))
+            } else {
+                gsp::LibosParams::from_chipset(chipset).wpr_heap_size(chipset, fb.end)?
+            };
             let wpr2_heap_addr = (elf.start - wpr2_heap_size).align_down(WPR2_HEAP_DOWN_ALIGN);
 
             FbRange(wpr2_heap_addr..(elf.start).align_down(WPR2_HEAP_DOWN_ALIGN))
@@ -265,7 +280,7 @@ pub(crate) fn new(chipset: Chipset, bar: Bar0<'_>, gsp_fw: &GspFirmware) -> Resu
             wpr2_heap,
             wpr2,
             heap,
-            vf_partition_count: 0,
+            vf_partition_count,
             pmu_reserved_size: hal.pmu_reserved_size(),
         })
     }
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index 6821008d48d9..a159f36bf704 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -26,6 +26,7 @@
 mod sequencer;
 
 pub(crate) use fw::{
+    vgpu_fw_heap_size,
     GspFmcBootParams,
     GspFwWprMeta,
     LibosParams, //
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index c607081e8242..dea028f4b434 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -56,7 +56,7 @@ pub(crate) fn boot(
 
         let gsp_fw = KBox::pin_init(GspFirmware::new(dev, chipset, FIRMWARE_VERSION), GFP_KERNEL)?;
 
-        let fb_layout = FbLayout::new(chipset, bar, &gsp_fw)?;
+        let fb_layout = FbLayout::new(chipset, bar, &gsp_fw, ctx.vgpu)?;
         dev_dbg!(dev, "{:#x?}\n", fb_layout);
 
         let wpr_meta = Coherent::init(dev, GFP_KERNEL, GspFwWprMeta::new(&gsp_fw, &fb_layout))?;
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index 4db0cfa4dc4d..2fb72f8dc3c9 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -101,6 +101,16 @@ pub(in crate::gsp) fn advance_cpu_write_ptr(qs: &Coherent<GspMem>, count: u32) {
 pub(crate) const GSP_MSG_QUEUE_ELEMENT_SIZE_MAX: usize =
     num::u32_as_usize(bindings::GSP_MSG_QUEUE_ELEMENT_SIZE_MAX);
 
+const GSP_FW_HEAP_SIZE_VGPU_DEFAULT: u64 = 581 * u64::SZ_1M;
+const GSP_FW_HEAP_SIZE_VGPU_48VMS: u64 = 1370 * u64::SZ_1M;
+
+pub(crate) fn vgpu_fw_heap_size(total_vfs: u32) -> u64 {
+    match total_vfs {
+        2..=32 => GSP_FW_HEAP_SIZE_VGPU_DEFAULT,
+        _ => GSP_FW_HEAP_SIZE_VGPU_48VMS,
+    }
+}
+
 /// Empty type to group methods related to heap parameters for running the GSP firmware.
 enum GspFwHeapParams {}
 
-- 
2.51.0

Re: [PATCH v2 1/7] PCI/IOV: Return u16 from pci_sriov_get_totalvfs()

[Alexandre Courbot]

On Tue Jun 23, 2026 at 4:43 AM JST, Zhi Wang wrote:
> pci_sriov_get_totalvfs() reports a VF count, not an errno-style
> status. It returns 0 when SR-IOV is unavailable or the device is not a
> PF, and otherwise returns the PF's driver_max_VFs value.
>
> driver_max_VFs is stored as a u16 in struct pci_sriov. It is derived
> from the SR-IOV TotalVFs field or from a driver-provided limit, so the
> implementation cannot return a negative value.
>
> Change the declaration, CONFIG_PCI_IOV stub, and implementation to
> return u16. Update callers to store the result in u16 variables, remove
> obsolete negative-value checks, and use unsigned format specifiers where
> needed.
>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: linux-pci@vger.kernel.org
> Signed-off-by: Zhi Wang <zhiw@nvidia.com>

Suggested-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://lore.kernel.org/all/DETDILPA1GFY.27WND0TEC5352@nvidia.com/

> ---
>  drivers/crypto/hisilicon/qm.c                   | 8 +++++---
>  drivers/crypto/intel/qat/qat_common/adf_sriov.c | 6 +++---
>  drivers/gpu/drm/xe/xe_sriov_pf.c                | 6 ++----
>  drivers/misc/genwqe/card_base.c                 | 6 ++----
>  drivers/net/ethernet/cavium/thunder/nic_main.c  | 2 +-
>  drivers/net/ethernet/emulex/benet/be_main.c     | 3 ++-
>  drivers/net/ethernet/mellanox/mlx5/core/sriov.c | 3 ++-
>  drivers/net/ethernet/sfc/ef10_sriov.c           | 2 +-

I believe that you can avoid converting all these drivers in this patch.
The implicit `u16 -> int` conversion done by C should result in the
expected behavior, and it will be fewer Acked-by to collect.

I.e. just updating drivers/pci/iov.c and include/linux/pci.h should be
sufficient as the first step. Specific drivers can then be updated using
separate patches that will be easier to merge individually, if you want
to do so.

Re: [PATCH v2 1/7] PCI/IOV: Return u16 from pci_sriov_get_totalvfs()

[David Laight]

On Wed, 24 Jun 2026 21:40:52 +0900
"Alexandre Courbot" <acourbot@nvidia.com> wrote:

> On Tue Jun 23, 2026 at 4:43 AM JST, Zhi Wang wrote:
> > pci_sriov_get_totalvfs() reports a VF count, not an errno-style
> > status. It returns 0 when SR-IOV is unavailable or the device is not a
> > PF, and otherwise returns the PF's driver_max_VFs value.
> >
> > driver_max_VFs is stored as a u16 in struct pci_sriov. It is derived
> > from the SR-IOV TotalVFs field or from a driver-provided limit, so the
> > implementation cannot return a negative value.
> >
> > Change the declaration, CONFIG_PCI_IOV stub, and implementation to
> > return u16. Update callers to store the result in u16 variables, remove
> > obsolete negative-value checks, and use unsigned format specifiers where
> > needed.
> >
> > Cc: Bjorn Helgaas <bhelgaas@google.com>
> > Cc: linux-pci@vger.kernel.org
> > Signed-off-by: Zhi Wang <zhiw@nvidia.com>  
> 
> Suggested-by: Alexandre Courbot <acourbot@nvidia.com>
> Link: https://lore.kernel.org/all/DETDILPA1GFY.27WND0TEC5352@nvidia.com/
> 
> > ---
> >  drivers/crypto/hisilicon/qm.c                   | 8 +++++---
> >  drivers/crypto/intel/qat/qat_common/adf_sriov.c | 6 +++---
> >  drivers/gpu/drm/xe/xe_sriov_pf.c                | 6 ++----
> >  drivers/misc/genwqe/card_base.c                 | 6 ++----
> >  drivers/net/ethernet/cavium/thunder/nic_main.c  | 2 +-
> >  drivers/net/ethernet/emulex/benet/be_main.c     | 3 ++-
> >  drivers/net/ethernet/mellanox/mlx5/core/sriov.c | 3 ++-
> >  drivers/net/ethernet/sfc/ef10_sriov.c           | 2 +-  
> 
> I believe that you can avoid converting all these drivers in this patch.
> The implicit `u16 -> int` conversion done by C should result in the
> expected behavior, and it will be fewer Acked-by to collect.

The generated code is also likely to be slightly better if the function
return value is a 32bit value.

Similarly you don't really want to do any kind of maths on local variables
that aren't 32bit (or 64bit on 64bit builds).

The fact that the domain of a value fits in 16 bits doesn't mean that
it is better to use u16 - it is usually worse.
Pretty much the only place u16 should be used is to reduce the size
of structures.

So it is probably correct to change the return type to unsigned int and
remove the error return checks, but nothing else.

	David

> 
> I.e. just updating drivers/pci/iov.c and include/linux/pci.h should be
> sufficient as the first step. Specific drivers can then be updated using
> separate patches that will be easier to merge individually, if you want
> to do so.
>