* [PATCH v5 0/6] gpu: nova-core: boot GSP with vGPU enabled
@ 2026-07-22 7:39 Zhi Wang
2026-07-22 7:39 ` [PATCH v5 1/6] PCI/IOV: Return unsigned int from pci_sriov_get_totalvfs() Zhi Wang
` (5 more replies)
0 siblings, 6 replies; 9+ messages in thread
From: Zhi Wang @ 2026-07-22 7:39 UTC (permalink / raw)
To: dakr, acourbot
Cc: airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh,
lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney,
joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita,
aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang,
Zhi Wang
Booting GSP with vGPU enabled is part of the first milestone (M1)
together with the Rust fwctl abstraction [1] and the nova-core fwctl
driver [2] for upstream vGPU support. It allows us to validate the basic
GSP boot flow with vGPU enabled and upload vGPU types even before the
remaining nova-core dependencies are ready. The nova-vGPU WIP patches
for all milestones can be found at [3].
This version is based on the latest drm-rust-next, which now includes
Alexandre's v6 GSP boot process consolidation series [4].
Changes in v5:
- Rebase on the latest drm-rust-next.
- Move sriov_get_totalvfs() to the Device<Core> implementation and keep
vGPU state detection on the Core typestate. (Danilo)
- Preserve the VF count as NonZero<u16> in VgpuState and unwrap it only
at the firmware u8 conversion boundary. (Alex)
- Rename patch 4 to "gpu: nova-core: detect and store vGPU state". (Alex)
- Pass pdev directly to the device logging macros, retaining .as_ref()
only for the FSP method that takes Device<Bound>. (Alex)
Link to v4 [5]
Changes in v4:
- Rebase on the latest drm-rust-next and Alexandre's v6 GSP boot process
consolidation series.
- Add a conditional C helper for pci_sriov_get_totalvfs() so the Rust
abstraction also builds when CONFIG_PCI_IOV is disabled.
- Return Option<NonZero<u16>> from the Rust SR-IOV helper so callers must
handle the unavailable-SR-IOV case explicitly.
- Derive Debug for VgpuState and log the complete state when
VgpuManager is constructed.
- Warn and fall back to VgpuState::Disabled when the optional vGPU state
cannot be detected, so a detection failure does not prevent a
bare-metal probe.
- Document vGPU state detection, split the vGPU HAL implementations into
per-chip modules, and call the HAL method directly through the exposed
trait and selector.
v3:
- Split the FSP response header rename from the FSP PRC vGPU mode query
into a separate patch.
- Change pci_sriov_get_totalvfs() to return unsigned int on the C side
while keeping the Rust helper as u16.
- Move the vGPU capability gate into a dedicated vgpu::hal module.
- Represent detected vGPU state as VgpuState instead of separate
enabled/total_vfs accessors.
- Keep total_vfs values below 2 on the disabled path, with an explicit
comment for the current single-VF limitation.
- Use the generated 570.144 GSP_FW_HEAP_SIZE_VGPU_DEFAULT binding for
vGPU WPR2 heap sizing, and keep unsupported 0/1-VF states out of the
vGPU heap path through VgpuState.
v2:
- Rebase on top of Alexandre's GSP boot process consolidation series.
- Drop the FSP response, FSP documentation, and GspBootContext patches
that are already in drm-rust-next or superseded by the prerequisite
boot consolidation series.
- Change pci_sriov_get_totalvfs() to return u16 and update existing C
callers accordingly.
- Make the Rust sriov_get_totalvfs() helper return u16 directly.
- Rework the FSP PRC vGPU mode query to use typed subcommand, object ID,
flags, request, and response structures.
- Move vGPU state detection before GSP boot into a read-only VgpuManager,
avoiding Mutex/Cell based mutation during boot.
- Add a HAL method for the vGPU capability gate.
- Split the SetRegistry changes into a dynamic-entry refactor and the
RMSetSriovMode functional change.
- Rework WPR2 heap sizing to consume VgpuManager, keep the vGPU heap-size
helper in gsp/fw.rs, and drop the 1VM heap-size special case.
[1] https://lore.kernel.org/rust-for-linux/20260217204909.211793-1-zhiw@nvidia.com/
[2] https://lore.kernel.org/rust-for-linux/20260305190936.398590-1-zhiw@nvidia.com/
[3] https://github.com/zhiwang-nvidia/nova-core/tree/zhi/nova-vgpu-wip
[4] https://lore.kernel.org/all/20260709-nova-bootcontext-v6-0-520cbf8b9b50@nvidia.com/
[5] https://lore.kernel.org/nova-gpu/20260709150206.1046839-1-zhiw@nvidia.com/
Zhi Wang (6):
PCI/IOV: Return unsigned int from pci_sriov_get_totalvfs()
rust: pci: add sriov_get_totalvfs() helper
gpu: nova-core: read vGPU mode from FSP via PRC protocol
gpu: nova-core: detect and store vGPU state
gpu: nova-core: set RMSetSriovMode for vGPU
gpu: nova-core: reserve vGPU WPR2 heap
drivers/gpu/nova-core/fb.rs | 25 ++-
drivers/gpu/nova-core/fsp.rs | 172 ++++++++++++++++++
drivers/gpu/nova-core/gpu.rs | 7 +
drivers/gpu/nova-core/gsp.rs | 2 +
drivers/gpu/nova-core/gsp/boot.rs | 4 +-
drivers/gpu/nova-core/gsp/commands.rs | 14 +-
drivers/gpu/nova-core/gsp/fw.rs | 5 +
.../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 1 +
drivers/gpu/nova-core/mctp.rs | 2 +
drivers/gpu/nova-core/nova_core.rs | 1 +
drivers/gpu/nova-core/vgpu.rs | 91 +++++++++
drivers/gpu/nova-core/vgpu/hal.rs | 25 +++
drivers/gpu/nova-core/vgpu/hal/gb202.rs | 15 ++
drivers/gpu/nova-core/vgpu/hal/tu102.rs | 15 ++
drivers/pci/iov.c | 2 +-
include/linux/pci.h | 4 +-
rust/helpers/pci.c | 8 +
rust/kernel/pci.rs | 13 ++
18 files changed, 395 insertions(+), 11 deletions(-)
create mode 100644 drivers/gpu/nova-core/vgpu.rs
create mode 100644 drivers/gpu/nova-core/vgpu/hal.rs
create mode 100644 drivers/gpu/nova-core/vgpu/hal/gb202.rs
create mode 100644 drivers/gpu/nova-core/vgpu/hal/tu102.rs
base-commit: 22e77d81d0a9ab3aee1c5538b3f2f8a66930bfa7
--
2.53.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v5 1/6] PCI/IOV: Return unsigned int from pci_sriov_get_totalvfs()
2026-07-22 7:39 [PATCH v5 0/6] gpu: nova-core: boot GSP with vGPU enabled Zhi Wang
@ 2026-07-22 7:39 ` Zhi Wang
2026-07-22 7:39 ` [PATCH v5 2/6] rust: pci: add sriov_get_totalvfs() helper Zhi Wang
` (4 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Zhi Wang @ 2026-07-22 7:39 UTC (permalink / raw)
To: dakr, acourbot
Cc: airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh,
lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney,
joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita,
aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang,
Zhi Wang, Bjorn Helgaas, David Laight, linux-pci
pci_sriov_get_totalvfs() reports a VF count, not an errno-style
status. It returns 0 when SR-IOV is unavailable or the device is not a
PF, and otherwise returns the PF's driver_max_VFs value.
driver_max_VFs is stored as a u16 in struct pci_sriov. It is derived
from the SR-IOV TotalVFs field or from a driver-provided limit, so the
implementation cannot return a negative value.
Change the declaration, CONFIG_PCI_IOV stub, and implementation to
return unsigned int.
Suggested-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Gary Guo <gary@garyguo.net>
Cc: linux-pci@vger.kernel.org
Link: https://lore.kernel.org/all/DJHPRE4TGGT8.BUTMYOF5YE05@nvidia.com/
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/pci/iov.c | 2 +-
include/linux/pci.h | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index b0d24839c084..9d408fb8ac25 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -1283,7 +1283,7 @@ EXPORT_SYMBOL_GPL(pci_sriov_set_totalvfs);
* SRIOV capability value of TotalVFs or the value of driver_max_VFs
* if the driver reduced it. Otherwise 0.
*/
-int pci_sriov_get_totalvfs(struct pci_dev *dev)
+unsigned int pci_sriov_get_totalvfs(struct pci_dev *dev)
{
if (!dev->is_physfn)
return 0;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index ebb5b9d76360..2b9c61de5f67 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -2569,7 +2569,7 @@ void pci_iov_remove_virtfn(struct pci_dev *dev, int id);
int pci_num_vf(struct pci_dev *dev);
int pci_vfs_assigned(struct pci_dev *dev);
int pci_sriov_set_totalvfs(struct pci_dev *dev, u16 numvfs);
-int pci_sriov_get_totalvfs(struct pci_dev *dev);
+unsigned int pci_sriov_get_totalvfs(struct pci_dev *dev);
int pci_sriov_configure_simple(struct pci_dev *dev, int nr_virtfn);
resource_size_t pci_iov_resource_size(const struct pci_dev *dev, int resno);
int pci_iov_vf_bar_set_size(struct pci_dev *dev, int resno, int size);
@@ -2622,7 +2622,7 @@ static inline int pci_vfs_assigned(struct pci_dev *dev)
{ return 0; }
static inline int pci_sriov_set_totalvfs(struct pci_dev *dev, u16 numvfs)
{ return 0; }
-static inline int pci_sriov_get_totalvfs(struct pci_dev *dev)
+static inline unsigned int pci_sriov_get_totalvfs(struct pci_dev *dev)
{ return 0; }
#define pci_sriov_configure_simple NULL
static inline resource_size_t pci_iov_resource_size(const struct pci_dev *dev,
--
2.53.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v5 2/6] rust: pci: add sriov_get_totalvfs() helper
2026-07-22 7:39 [PATCH v5 0/6] gpu: nova-core: boot GSP with vGPU enabled Zhi Wang
2026-07-22 7:39 ` [PATCH v5 1/6] PCI/IOV: Return unsigned int from pci_sriov_get_totalvfs() Zhi Wang
@ 2026-07-22 7:39 ` Zhi Wang
2026-07-22 7:39 ` [PATCH v5 3/6] gpu: nova-core: read vGPU mode from FSP via PRC protocol Zhi Wang
` (3 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Zhi Wang @ 2026-07-22 7:39 UTC (permalink / raw)
To: dakr, acourbot
Cc: airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh,
lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney,
joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita,
aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang,
Zhi Wang, Bjorn Helgaas, David Laight, linux-pci
Expose pci_sriov_get_totalvfs() to Rust PCI drivers so they can query
how many SR-IOV VFs a device supports.
Use a conditional C helper because the !CONFIG_PCI_IOV version of
pci_sriov_get_totalvfs() is a static inline function and is therefore
not emitted into the Rust bindings. Return Option<NonZero<u16>> so Rust
callers must handle the zero value that represents unavailable SR-IOV.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Cc: Alexandre Courbot <acourbot@nvidia.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Gary Guo <gary@garyguo.net>
Cc: linux-pci@vger.kernel.org
Link: https://lore.kernel.org/all/DJHPRE4TGGT8.BUTMYOF5YE05@nvidia.com/
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
rust/helpers/pci.c | 8 ++++++++
rust/kernel/pci.rs | 13 +++++++++++++
2 files changed, 21 insertions(+)
diff --git a/rust/helpers/pci.c b/rust/helpers/pci.c
index e44905317d75..4ebf256dff23 100644
--- a/rust/helpers/pci.c
+++ b/rust/helpers/pci.c
@@ -24,6 +24,14 @@ __rust_helper bool rust_helper_dev_is_pci(const struct device *dev)
return dev_is_pci(dev);
}
+#ifndef CONFIG_PCI_IOV
+__rust_helper unsigned int
+rust_helper_pci_sriov_get_totalvfs(struct pci_dev *pdev)
+{
+ return pci_sriov_get_totalvfs(pdev);
+}
+#endif
+
#ifndef CONFIG_PCI_MSI
__rust_helper int rust_helper_pci_alloc_irq_vectors(struct pci_dev *dev,
unsigned int min_vecs,
diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index c6d6bd8f251d..9f19ccd5905c 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -25,6 +25,7 @@
use core::{
marker::PhantomData,
mem::offset_of,
+ num::NonZero,
ptr::{
addr_of_mut,
NonNull, //
@@ -452,6 +453,18 @@ pub fn pci_class(&self) -> Class {
}
impl<'a> Device<device::Core<'a>> {
+ /// Returns the total number of VFs, or [`None`] if SR-IOV is not available.
+ #[inline]
+ pub fn sriov_get_totalvfs(&self) -> Option<NonZero<u16>> {
+ // SAFETY: `self.as_raw()` is a valid pointer to a `struct pci_dev`.
+ let total_vfs = unsafe { bindings::pci_sriov_get_totalvfs(self.as_raw()) };
+
+ // CAST: The C function returns `unsigned int`, but the value originates
+ // from TotalVFs/driver_max_VFs (which are defined as `u16`), so this cast
+ // cannot truncate.
+ NonZero::new(total_vfs as u16)
+ }
+
/// Enable memory resources for this device.
pub fn enable_device_mem(&self) -> Result {
// SAFETY: `self.as_raw` is guaranteed to be a pointer to a valid `struct pci_dev`.
--
2.53.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v5 3/6] gpu: nova-core: read vGPU mode from FSP via PRC protocol
2026-07-22 7:39 [PATCH v5 0/6] gpu: nova-core: boot GSP with vGPU enabled Zhi Wang
2026-07-22 7:39 ` [PATCH v5 1/6] PCI/IOV: Return unsigned int from pci_sriov_get_totalvfs() Zhi Wang
2026-07-22 7:39 ` [PATCH v5 2/6] rust: pci: add sriov_get_totalvfs() helper Zhi Wang
@ 2026-07-22 7:39 ` Zhi Wang
2026-07-22 7:39 ` [PATCH v5 4/6] gpu: nova-core: detect and store vGPU state Zhi Wang
` (2 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Zhi Wang @ 2026-07-22 7:39 UTC (permalink / raw)
To: dakr, acourbot
Cc: airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh,
lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney,
joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita,
aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang,
Zhi Wang
vGPU boot needs to know whether firmware reports vGPU mode as active.
FSP's Management Partition exposes PRC (Product Reconfiguration Control)
as an API for reading device configuration knobs without firmware
updates. The vGPU mode knob is one such configuration value.
Add typed PRC request and response payloads for the vGPU mode object,
add the PRC NVDM type, and parse the returned knob value into VgpuMode.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/fsp.rs | 173 ++++++++++++++++++++++++++++++++++
drivers/gpu/nova-core/mctp.rs | 2 +
2 files changed, 175 insertions(+)
diff --git a/drivers/gpu/nova-core/fsp.rs b/drivers/gpu/nova-core/fsp.rs
index afbd75879d16..3a6b02f0dc11 100644
--- a/drivers/gpu/nova-core/fsp.rs
+++ b/drivers/gpu/nova-core/fsp.rs
@@ -52,6 +52,56 @@
mod hal;
+/// PRC message sub-command.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+#[repr(u8)]
+enum PrcMessageSubcmd {
+ /// Read a PRC knob value.
+ Read = 0x0c,
+}
+
+impl From<PrcMessageSubcmd> for u8 {
+ fn from(value: PrcMessageSubcmd) -> Self {
+ value as u8
+ }
+}
+
+/// PRC object identifier.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+#[repr(u8)]
+enum PrcObjectId {
+ /// vGPU mode configuration knob.
+ VgpuMode = 0x29,
+}
+
+impl From<PrcObjectId> for u8 {
+ fn from(value: PrcObjectId) -> Self {
+ value as u8
+ }
+}
+
+kernel::impl_flags!(
+ /// PRC request flags.
+ #[derive(Clone, Copy, Default, PartialEq, Eq)]
+ struct PrcFlags(u8);
+
+ /// Individual PRC request flag.
+ #[derive(Clone, Copy, PartialEq, Eq)]
+ enum PrcFlag {
+ /// Request the active knob value for the current boot.
+ Active = 1 << 1,
+ }
+);
+
+/// vGPU operating mode as reported by FSP via the PRC protocol.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum VgpuMode {
+ /// vGPU support is disabled on this GPU.
+ Disabled,
+ /// vGPU support is enabled on this GPU.
+ Enabled,
+}
+
/// FSP command response payload (`NVDM_PAYLOAD_COMMAND_RESPONSE`).
#[repr(C, packed)]
#[derive(Clone, Copy)]
@@ -61,6 +111,62 @@ struct NvdmPayloadCommandResponse {
error_code: u32,
}
+/// PRC message payload.
+///
+/// Sent to FSP to query or modify a device configuration knob.
+#[repr(C, packed)]
+#[derive(Clone, Copy)]
+struct NvdmPayloadPrc {
+ sub_message_id: u8,
+ flags: u8,
+ object_id: u8,
+ reserved: u8,
+}
+
+impl NvdmPayloadPrc {
+ /// Constructs a PRC payload from typed protocol fields.
+ fn new(subcmd: PrcMessageSubcmd, object_id: PrcObjectId, flags: PrcFlags) -> Self {
+ Self {
+ sub_message_id: subcmd.into(),
+ flags: flags.into(),
+ object_id: object_id.into(),
+ reserved: 0,
+ }
+ }
+}
+
+// SAFETY: NvdmPayloadPrc is a packed C struct with only integral fields.
+unsafe impl AsBytes for NvdmPayloadPrc {}
+
+/// PRC response payload containing the knob state value.
+#[repr(C, packed)]
+#[derive(Clone, Copy)]
+struct NvdmPayloadPrcResponse {
+ value_low: u8,
+ value_high: u8,
+ reserved1: u8,
+ reserved2: u8,
+}
+
+impl NvdmPayloadPrcResponse {
+ /// Returns the PRC knob value as a little-endian 16-bit integer.
+ fn value(self) -> u16 {
+ u16::from(self.value_low) | (u16::from(self.value_high) << 8)
+ }
+}
+
+impl TryFrom<NvdmPayloadPrcResponse> for VgpuMode {
+ type Error = kernel::error::Error;
+
+ fn try_from(value: NvdmPayloadPrcResponse) -> Result<Self> {
+ match value.value() {
+ 0 => Ok(VgpuMode::Disabled),
+ 1 => Ok(VgpuMode::Enabled),
+ _ => Err(EINVAL),
+ }
+ }
+}
+
/// Common MCTP and NVDM headers shared by all FSP messages.
#[repr(C, packed)]
#[derive(Clone, Copy)]
@@ -96,6 +202,17 @@ struct FspResponseHeader {
// SAFETY: FspResponseHeader is a packed C struct with only integral fields.
unsafe impl FromBytes for FspResponseHeader {}
+/// Complete FSP PRC response including the knob state payload.
+#[repr(C, packed)]
+#[derive(Clone, Copy)]
+struct FspPrcResponse {
+ header: FspResponseHeader,
+ prc_data: NvdmPayloadPrcResponse,
+}
+
+// SAFETY: FspPrcResponse is a packed C struct with only integral fields.
+unsafe impl FromBytes for FspPrcResponse {}
+
/// Trait implemented by types representing a message to send to FSP.
///
/// This provides [`Fsp::send_sync_fsp`] with the information it needs to send
@@ -183,10 +300,35 @@ fn new<'a>(
// bytes are initialized.
unsafe impl AsBytes for FspCotMessage {}
+/// Complete FSP PRC message.
+#[repr(C, packed)]
+#[derive(Clone, Copy)]
+struct FspPrcMessage {
+ header: FspMessageHeader,
+ prc: NvdmPayloadPrc,
+}
+
+impl FspPrcMessage {
+ /// Constructs a PRC message.
+ fn new(subcmd: PrcMessageSubcmd, object_id: PrcObjectId, flags: PrcFlags) -> Self {
+ Self {
+ header: FspMessageHeader::new(NvdmType::Prc),
+ prc: NvdmPayloadPrc::new(subcmd, object_id, flags),
+ }
+ }
+}
+
+// SAFETY: FspPrcMessage is a packed C struct with only integral fields.
+unsafe impl AsBytes for FspPrcMessage {}
+
impl MessageToFsp for FspCotMessage {
const NVDM_TYPE: NvdmType = NvdmType::Cot;
}
+impl MessageToFsp for FspPrcMessage {
+ const NVDM_TYPE: NvdmType = NvdmType::Prc;
+}
+
/// Bundled arguments for FMC boot via FSP Chain of Trust.
pub(crate) struct FmcBootArgs {
chipset: Chipset,
@@ -342,6 +484,37 @@ fn send_sync_fsp<M>(&mut self, dev: &device::Device, msg: &M) -> Result<KVec<u8>
Ok(response_buf)
}
+ /// Reads the active vGPU mode from FSP using the PRC protocol.
+ ///
+ /// Queries FSP's Management Partition for the active vGPU mode knob value.
+ #[expect(dead_code)]
+ pub(crate) fn read_vgpu_mode(
+ &mut self,
+ dev: &device::Device<device::Bound>,
+ ) -> Result<VgpuMode> {
+ let msg = KBox::new(
+ FspPrcMessage::new(
+ PrcMessageSubcmd::Read,
+ PrcObjectId::VgpuMode,
+ PrcFlags::from(PrcFlag::Active),
+ ),
+ GFP_KERNEL,
+ )?;
+
+ let response_buf = self.send_sync_fsp(dev, &*msg)?;
+ let (prc_response, _) =
+ FspPrcResponse::from_bytes_prefix(&response_buf[..]).ok_or_else(|| {
+ dev_err!(dev, "PRC response too small: {}\n", response_buf.len());
+ EIO
+ })?;
+
+ let prc_data = prc_response.prc_data;
+
+ VgpuMode::try_from(prc_data).inspect_err(|_| {
+ dev_err!(dev, "Unexpected vGPU mode value: {:#x}\n", prc_data.value());
+ })
+ }
+
/// Boots GSP FMC via FSP Chain of Trust.
///
/// Builds the CoT message from the pre-configured [`FmcBootArgs`], sends it
diff --git a/drivers/gpu/nova-core/mctp.rs b/drivers/gpu/nova-core/mctp.rs
index acc2abbd4b0c..90c642c91a72 100644
--- a/drivers/gpu/nova-core/mctp.rs
+++ b/drivers/gpu/nova-core/mctp.rs
@@ -22,6 +22,8 @@
/// NVDM message type identifiers carried over MCTP.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum NvdmType with TryFrom<Bounded<u32, 8>> {
+ /// PRC (Product Reconfiguration Control) message.
+ Prc = 0x13,
/// Chain of Trust boot message.
Cot = 0x14,
/// FSP command response.
--
2.53.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v5 4/6] gpu: nova-core: detect and store vGPU state
2026-07-22 7:39 [PATCH v5 0/6] gpu: nova-core: boot GSP with vGPU enabled Zhi Wang
` (2 preceding siblings ...)
2026-07-22 7:39 ` [PATCH v5 3/6] gpu: nova-core: read vGPU mode from FSP via PRC protocol Zhi Wang
@ 2026-07-22 7:39 ` Zhi Wang
2026-07-22 7:39 ` [PATCH v5 5/6] gpu: nova-core: set RMSetSriovMode for vGPU Zhi Wang
2026-07-22 7:39 ` [PATCH v5 6/6] gpu: nova-core: reserve vGPU WPR2 heap Zhi Wang
5 siblings, 0 replies; 9+ messages in thread
From: Zhi Wang @ 2026-07-22 7:39 UTC (permalink / raw)
To: dakr, acourbot
Cc: airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh,
lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney,
joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita,
aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang,
Zhi Wang
GSP boot needs a stable view of vGPU state before it starts building the
boot-time data structures that depend on SR-IOV and firmware policy. That
state must be derived once from the PCI VF count and the FSP PRC vGPU mode
knob before booting GSP.
Add VgpuManager to detect and retain the vGPU state during GPU
construction. Keep the manager separate from the detected state because
later vGPU milestones will add vGPU resources and lifecycle state to it.
Keep the vGPU capability gate local to the vGPU module with per-chip HAL
modules. Treat failures to detect the optional vGPU state as disabled so
they do not prevent a bare-metal probe, and log both the failure and the
detected state where the manager is constructed.
Cc: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/fsp.rs | 1 -
drivers/gpu/nova-core/gpu.rs | 7 ++
drivers/gpu/nova-core/gsp.rs | 3 +
drivers/gpu/nova-core/nova_core.rs | 1 +
drivers/gpu/nova-core/vgpu.rs | 93 +++++++++++++++++++++++++
drivers/gpu/nova-core/vgpu/hal.rs | 25 +++++++
drivers/gpu/nova-core/vgpu/hal/gb202.rs | 15 ++++
drivers/gpu/nova-core/vgpu/hal/tu102.rs | 15 ++++
8 files changed, 159 insertions(+), 1 deletion(-)
create mode 100644 drivers/gpu/nova-core/vgpu.rs
create mode 100644 drivers/gpu/nova-core/vgpu/hal.rs
create mode 100644 drivers/gpu/nova-core/vgpu/hal/gb202.rs
create mode 100644 drivers/gpu/nova-core/vgpu/hal/tu102.rs
diff --git a/drivers/gpu/nova-core/fsp.rs b/drivers/gpu/nova-core/fsp.rs
index 3a6b02f0dc11..abc161c4ba96 100644
--- a/drivers/gpu/nova-core/fsp.rs
+++ b/drivers/gpu/nova-core/fsp.rs
@@ -487,7 +487,6 @@ fn send_sync_fsp<M>(&mut self, dev: &device::Device, msg: &M) -> Result<KVec<u8>
/// Reads the active vGPU mode from FSP using the PRC protocol.
///
/// Queries FSP's Management Partition for the active vGPU mode knob value.
- #[expect(dead_code)]
pub(crate) fn read_vgpu_mode(
&mut self,
dev: &device::Device<device::Bound>,
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index 442c0979f9c6..42a4cd7971fa 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -30,6 +30,7 @@
GspBootContext, //
},
regs,
+ vgpu::VgpuManager, //
};
mod hal;
@@ -267,6 +268,8 @@ struct GspResources<'gpu> {
// TODO: use different resource types for each boot method, and make the relevant Gsp methods
// generic against them.
fsp: Option<Fsp<'gpu>>,
+ /// vGPU state detected before GSP boot.
+ vgpu: VgpuManager,
/// GSP runtime data.
#[pin]
gsp: Gsp,
@@ -311,6 +314,7 @@ fn drop(self: Pin<&mut Self>) {
gsp_falcon: &*this.gsp_falcon,
sec2_falcon: &*this.sec2_falcon,
fsp: this.fsp.as_mut(),
+ vgpu: &*this.vgpu,
},
bundle,
)
@@ -364,6 +368,8 @@ pub(crate) fn new(
fsp: Fsp::try_new(dev, bar, spec.chipset)?,
+ vgpu: VgpuManager::new(pdev, spec.chipset, fsp.as_mut()),
+
gsp <- Gsp::new(pdev),
// This member must be initialized last, so the `UnloadBundle` can never be dropped
@@ -376,6 +382,7 @@ pub(crate) fn new(
gsp_falcon,
sec2_falcon,
fsp: fsp.as_mut(),
+ vgpu,
})?,
}),
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index f38630026e5d..0480881d437f 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -51,6 +51,7 @@
},
},
num,
+ vgpu::VgpuManager, //
};
pub(crate) const GSP_PAGE_SHIFT: usize = 12;
@@ -69,6 +70,8 @@ pub(crate) struct GspBootContext<'ctx, 'gpu> {
pub(crate) gsp_falcon: &'ctx Falcon<'gpu, GspFalcon>,
pub(crate) sec2_falcon: &'ctx Falcon<'gpu, Sec2Falcon>,
pub(crate) fsp: Option<&'ctx mut Fsp<'gpu>>,
+ #[expect(dead_code)]
+ pub(crate) vgpu: &'ctx VgpuManager,
}
impl<'ctx, 'gpu> GspBootContext<'ctx, 'gpu> {
diff --git a/drivers/gpu/nova-core/nova_core.rs b/drivers/gpu/nova-core/nova_core.rs
index a61406ba5c0b..35a8b1214b0e 100644
--- a/drivers/gpu/nova-core/nova_core.rs
+++ b/drivers/gpu/nova-core/nova_core.rs
@@ -23,6 +23,7 @@
mod regs;
mod sbuffer;
mod vbios;
+mod vgpu;
pub(crate) const MODULE_NAME: &core::ffi::CStr = <LocalModule as kernel::ModuleMetadata>::NAME;
diff --git a/drivers/gpu/nova-core/vgpu.rs b/drivers/gpu/nova-core/vgpu.rs
new file mode 100644
index 000000000000..71fd0fe879c2
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu.rs
@@ -0,0 +1,93 @@
+// SPDX-License-Identifier: GPL-2.0
+
+use core::num::NonZero;
+
+use kernel::{
+ device,
+ pci,
+ prelude::*, //
+};
+
+use crate::{
+ fsp::{
+ Fsp,
+ VgpuMode, //
+ },
+ gpu::Chipset, //
+};
+
+mod hal;
+
+/// vGPU state detected during GPU construction.
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum VgpuState {
+ /// vGPU mode is not enabled for this boot.
+ Disabled,
+ /// vGPU mode is enabled for this boot.
+ Enabled {
+ /// Total number of SR-IOV VFs supported by this device.
+ #[expect(dead_code)]
+ total_vfs: NonZero<u16>,
+ },
+}
+
+/// vGPU state manager.
+pub(crate) struct VgpuManager {
+ state: VgpuState,
+}
+
+impl VgpuManager {
+ /// Creates a vGPU manager by querying SR-IOV and the FSP PRC vGPU knob.
+ pub(crate) fn new(
+ pdev: &pci::Device<device::Core<'_>>,
+ chipset: Chipset,
+ fsp: Option<&mut Fsp<'_>>,
+ ) -> Self {
+ let state = Self::detect_state(pdev, chipset, fsp).unwrap_or_else(|e| {
+ dev_warn!(
+ pdev,
+ "vGPU state detection failed: {:?}; disabling vGPU\n",
+ e
+ );
+ VgpuState::Disabled
+ });
+ dev_dbg!(pdev, "vGPU state: {:?}\n", state);
+
+ Self { state }
+ }
+
+ /// Detects the vGPU state from the chipset, SR-IOV capability and FSP PRC knob.
+ fn detect_state(
+ pdev: &pci::Device<device::Core<'_>>,
+ chipset: Chipset,
+ fsp: Option<&mut Fsp<'_>>,
+ ) -> Result<VgpuState> {
+ if !hal::vgpu_hal(chipset).supports_vgpu() {
+ return Ok(VgpuState::Disabled);
+ }
+
+ let Some(total_vfs) = pdev.sriov_get_totalvfs() else {
+ return Ok(VgpuState::Disabled);
+ };
+
+ if total_vfs.get() < 2 {
+ // The current vGPU path does not support single-VF SR-IOV devices yet.
+ // Treat one total VF as vGPU-disabled for now; single-VF support can relax
+ // this gate once the manager handles that topology.
+ return Ok(VgpuState::Disabled);
+ }
+
+ let fsp = fsp.ok_or(ENODEV)?;
+
+ match fsp.read_vgpu_mode(pdev.as_ref())? {
+ VgpuMode::Enabled => Ok(VgpuState::Enabled { total_vfs }),
+ VgpuMode::Disabled => Ok(VgpuState::Disabled),
+ }
+ }
+
+ /// Returns the detected vGPU state for this boot.
+ #[expect(dead_code)]
+ pub(crate) fn state(&self) -> VgpuState {
+ self.state
+ }
+}
diff --git a/drivers/gpu/nova-core/vgpu/hal.rs b/drivers/gpu/nova-core/vgpu/hal.rs
new file mode 100644
index 000000000000..456f8fe27582
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/hal.rs
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0
+
+use crate::gpu::{
+ Architecture,
+ Chipset, //
+};
+
+mod gb202;
+mod tu102;
+
+pub(super) trait VgpuHal {
+ /// Returns whether this chipset can support vGPU.
+ fn supports_vgpu(&self) -> bool;
+}
+
+pub(super) fn vgpu_hal(chipset: Chipset) -> &'static dyn VgpuHal {
+ match chipset.arch() {
+ Architecture::BlackwellGB20x => gb202::GB202_HAL,
+ Architecture::Turing
+ | Architecture::Ampere
+ | Architecture::Hopper
+ | Architecture::Ada
+ | Architecture::BlackwellGB10x => tu102::TU102_HAL,
+ }
+}
diff --git a/drivers/gpu/nova-core/vgpu/hal/gb202.rs b/drivers/gpu/nova-core/vgpu/hal/gb202.rs
new file mode 100644
index 000000000000..3add8af26616
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/hal/gb202.rs
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+use crate::vgpu::hal::VgpuHal;
+
+struct Gb202;
+
+impl VgpuHal for Gb202 {
+ fn supports_vgpu(&self) -> bool {
+ true
+ }
+}
+
+const GB202: Gb202 = Gb202;
+pub(super) const GB202_HAL: &dyn VgpuHal = &GB202;
diff --git a/drivers/gpu/nova-core/vgpu/hal/tu102.rs b/drivers/gpu/nova-core/vgpu/hal/tu102.rs
new file mode 100644
index 000000000000..baeea3ac5754
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/hal/tu102.rs
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+use crate::vgpu::hal::VgpuHal;
+
+struct Tu102;
+
+impl VgpuHal for Tu102 {
+ fn supports_vgpu(&self) -> bool {
+ false
+ }
+}
+
+const TU102: Tu102 = Tu102;
+pub(super) const TU102_HAL: &dyn VgpuHal = &TU102;
--
2.53.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v5 5/6] gpu: nova-core: set RMSetSriovMode for vGPU
2026-07-22 7:39 [PATCH v5 0/6] gpu: nova-core: boot GSP with vGPU enabled Zhi Wang
` (3 preceding siblings ...)
2026-07-22 7:39 ` [PATCH v5 4/6] gpu: nova-core: detect and store vGPU state Zhi Wang
@ 2026-07-22 7:39 ` Zhi Wang
2026-07-22 7:52 ` sashiko-bot
2026-07-22 7:39 ` [PATCH v5 6/6] gpu: nova-core: reserve vGPU WPR2 heap Zhi Wang
5 siblings, 1 reply; 9+ messages in thread
From: Zhi Wang @ 2026-07-22 7:39 UTC (permalink / raw)
To: dakr, acourbot
Cc: airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh,
lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney,
joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita,
aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang,
Zhi Wang
The GSP registry setup needs to advertise SR-IOV mode when nova-core boots
GSP for an enabled vGPU configuration. Without the registry entry, GSP-RM
is not told to initialize in the mode required by NVIDIA vGPU.
Append RMSetSriovMode to the SetRegistry command when the vGPU state
detected before GSP boot is enabled. Keep the existing registry entries
unchanged for non-vGPU boots.
Cc: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/gsp.rs | 1 -
drivers/gpu/nova-core/gsp/boot.rs | 2 +-
drivers/gpu/nova-core/gsp/commands.rs | 14 +++++++++++++-
drivers/gpu/nova-core/vgpu.rs | 1 -
4 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index 0480881d437f..70123dc54981 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -70,7 +70,6 @@ pub(crate) struct GspBootContext<'ctx, 'gpu> {
pub(crate) gsp_falcon: &'ctx Falcon<'gpu, GspFalcon>,
pub(crate) sec2_falcon: &'ctx Falcon<'gpu, Sec2Falcon>,
pub(crate) fsp: Option<&'ctx mut Fsp<'gpu>>,
- #[expect(dead_code)]
pub(crate) vgpu: &'ctx VgpuManager,
}
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 6b7d00205000..a2e8342e7760 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -89,7 +89,7 @@ pub(crate) fn boot(
self.cmdq
.send_command_no_wait(bar, commands::SetSystemInfo::new(pdev, chipset))?;
self.cmdq
- .send_command_no_wait(bar, commands::SetRegistry::new()?)?;
+ .send_command_no_wait(bar, commands::SetRegistry::new(ctx.vgpu.state())?)?;
hal.post_boot(&self, ctx, &gsp_fw)?;
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 08380de39048..134f34b19174 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -34,6 +34,7 @@
},
},
sbuffer::SBufferIter,
+ vgpu::VgpuState, //
};
/// The `GspSetSystemInfo` command.
@@ -72,7 +73,7 @@ pub(crate) struct SetRegistry {
impl SetRegistry {
/// Creates a new `SetRegistry` command, using a set of hardcoded entries.
- pub(crate) fn new() -> Result<Self> {
+ pub(crate) fn new(vgpu_state: VgpuState) -> Result<Self> {
let mut entries = KVec::new();
// RMSecBusResetEnable - enables PCI secondary bus reset
@@ -104,6 +105,17 @@ pub(crate) fn new() -> Result<Self> {
GFP_KERNEL,
)?;
+ if matches!(vgpu_state, VgpuState::Enabled { .. }) {
+ // RMSetSriovMode - required when vGPU is enabled.
+ entries.push(
+ RegistryEntry {
+ key: "RMSetSriovMode",
+ value: 1,
+ },
+ GFP_KERNEL,
+ )?;
+ }
+
Ok(Self { entries })
}
}
diff --git a/drivers/gpu/nova-core/vgpu.rs b/drivers/gpu/nova-core/vgpu.rs
index 71fd0fe879c2..5359847e2eb9 100644
--- a/drivers/gpu/nova-core/vgpu.rs
+++ b/drivers/gpu/nova-core/vgpu.rs
@@ -86,7 +86,6 @@ fn detect_state(
}
/// Returns the detected vGPU state for this boot.
- #[expect(dead_code)]
pub(crate) fn state(&self) -> VgpuState {
self.state
}
--
2.53.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v5 6/6] gpu: nova-core: reserve vGPU WPR2 heap
2026-07-22 7:39 [PATCH v5 0/6] gpu: nova-core: boot GSP with vGPU enabled Zhi Wang
` (4 preceding siblings ...)
2026-07-22 7:39 ` [PATCH v5 5/6] gpu: nova-core: set RMSetSriovMode for vGPU Zhi Wang
@ 2026-07-22 7:39 ` Zhi Wang
2026-07-22 7:52 ` sashiko-bot
5 siblings, 1 reply; 9+ messages in thread
From: Zhi Wang @ 2026-07-22 7:39 UTC (permalink / raw)
To: dakr, acourbot
Cc: airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh,
lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney,
joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita,
aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang,
Zhi Wang
GSP-RM needs a larger WPR2 heap when booting in vGPU mode. The heap size
is firmware-dependent, so it should come from the generated firmware
bindings instead of being open-coded in nova-core.
Pass the detected vGPU state into the framebuffer layout calculation. Keep
baremetal boots on the existing heap sizing path, and use the 570.144
vGPU default heap binding only when vGPU is enabled. The same state match
also sets the VF partition count, so disabled and invalid 0/1-VF states do
not enter the vGPU heap path.
Cc: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/fb.rs | 25 +++++++++++++++----
drivers/gpu/nova-core/gsp/boot.rs | 2 +-
drivers/gpu/nova-core/gsp/fw.rs | 5 ++++
.../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 1 +
drivers/gpu/nova-core/vgpu.rs | 1 -
5 files changed, 27 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/nova-core/fb.rs b/drivers/gpu/nova-core/fb.rs
index 273cff752fae..fee6a3e11dfb 100644
--- a/drivers/gpu/nova-core/fb.rs
+++ b/drivers/gpu/nova-core/fb.rs
@@ -23,7 +23,8 @@
firmware::gsp::GspFirmware,
gpu::Chipset,
gsp,
- num::FromSafeCast, //
+ num::FromSafeCast,
+ vgpu::VgpuState, //
};
mod hal;
@@ -171,7 +172,12 @@ pub(crate) struct FbLayout {
impl FbLayout {
/// Computes the FB layout for `chipset` required to run the `gsp_fw` GSP firmware.
- pub(crate) fn new(chipset: Chipset, bar: Bar0<'_>, gsp_fw: &GspFirmware) -> Result<Self> {
+ pub(crate) fn new(
+ chipset: Chipset,
+ bar: Bar0<'_>,
+ gsp_fw: &GspFirmware,
+ vgpu_state: VgpuState,
+ ) -> Result<Self> {
let hal = hal::fb_hal(chipset);
let fb = {
@@ -234,10 +240,19 @@ pub(crate) fn new(chipset: Chipset, bar: Bar0<'_>, gsp_fw: &GspFirmware) -> Resu
FbRange(elf_addr..elf_addr + elf_size)
};
+ let (vf_partition_count, wpr2_heap_size) = match vgpu_state {
+ VgpuState::Disabled => (
+ 0,
+ gsp::LibosParams::from_chipset(chipset).wpr_heap_size(chipset, fb.end)?,
+ ),
+ VgpuState::Enabled { total_vfs } => (
+ u8::try_from(total_vfs.get()).map_err(|_| EINVAL)?,
+ gsp::LibosParams::vgpu_wpr_heap_size(),
+ ),
+ };
+
let wpr2_heap = {
const WPR2_HEAP_DOWN_ALIGN: Alignment = Alignment::new::<SZ_1M>();
- let wpr2_heap_size =
- gsp::LibosParams::from_chipset(chipset).wpr_heap_size(chipset, fb.end)?;
let wpr2_heap_addr = (elf.start - wpr2_heap_size).align_down(WPR2_HEAP_DOWN_ALIGN);
FbRange(wpr2_heap_addr..(elf.start).align_down(WPR2_HEAP_DOWN_ALIGN))
@@ -265,7 +280,7 @@ pub(crate) fn new(chipset: Chipset, bar: Bar0<'_>, gsp_fw: &GspFirmware) -> Resu
wpr2_heap,
wpr2,
heap,
- vf_partition_count: 0,
+ vf_partition_count,
pmu_reserved_size: hal.pmu_reserved_size(),
})
}
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index a2e8342e7760..727b8ae4bcb7 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -50,7 +50,7 @@ pub(crate) fn boot(
let gsp_fw = KBox::pin_init(GspFirmware::new(dev, chipset, FIRMWARE_VERSION), GFP_KERNEL)?;
- let fb_layout = FbLayout::new(chipset, bar, &gsp_fw)?;
+ let fb_layout = FbLayout::new(chipset, bar, &gsp_fw, ctx.vgpu.state())?;
dev_dbg!(dev, "{:#x?}\n", fb_layout);
let wpr_meta = Coherent::init(dev, GFP_KERNEL, GspFwWprMeta::new(&gsp_fw, &fb_layout))?;
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index 166b0c69edb7..6e8e7d822ef1 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -132,6 +132,11 @@ pub(crate) fn from_chipset(chipset: Chipset) -> &'static LibosParams {
}
}
+ /// Returns the WPR heap size to reserve when vGPU is enabled.
+ pub(crate) fn vgpu_wpr_heap_size() -> u64 {
+ u64::from(bindings::GSP_FW_HEAP_SIZE_VGPU_DEFAULT)
+ }
+
/// Returns the amount of memory (in bytes) to allocate for the WPR heap for a framebuffer size
/// of `fb_size` (in bytes) for `chipset`.
pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> Result<u64> {
diff --git a/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs b/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs
index ea350f9b2cc4..afe3e007f088 100644
--- a/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs
+++ b/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs
@@ -40,6 +40,7 @@ fn fmt(&self, fmt: &mut ::core::fmt::Formatter<'_>) -> ::core::fmt::Result {
pub const GSP_FW_HEAP_PARAM_BASE_RM_SIZE_GH100: u32 = 14680064;
pub const GSP_FW_HEAP_PARAM_SIZE_PER_GB_FB: u32 = 98304;
pub const GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE: u32 = 100663296;
+pub const GSP_FW_HEAP_SIZE_VGPU_DEFAULT: u32 = 609222656;
pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MIN_MB: u32 = 64;
pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB: u32 = 256;
pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MIN_MB: u32 = 88;
diff --git a/drivers/gpu/nova-core/vgpu.rs b/drivers/gpu/nova-core/vgpu.rs
index 5359847e2eb9..6b7e045acea8 100644
--- a/drivers/gpu/nova-core/vgpu.rs
+++ b/drivers/gpu/nova-core/vgpu.rs
@@ -26,7 +26,6 @@ pub(crate) enum VgpuState {
/// vGPU mode is enabled for this boot.
Enabled {
/// Total number of SR-IOV VFs supported by this device.
- #[expect(dead_code)]
total_vfs: NonZero<u16>,
},
}
--
2.53.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v5 6/6] gpu: nova-core: reserve vGPU WPR2 heap
2026-07-22 7:39 ` [PATCH v5 6/6] gpu: nova-core: reserve vGPU WPR2 heap Zhi Wang
@ 2026-07-22 7:52 ` sashiko-bot
0 siblings, 0 replies; 9+ messages in thread
From: sashiko-bot @ 2026-07-22 7:52 UTC (permalink / raw)
To: acourbot, dakr, Zhi Wang
Cc: ecourtney, joelagnelf, Alexandre Courbot, boqun.feng, zhiwang,
linux-kernel, a.hindborg, apopple, cjia, smitra, airlied, ankita,
aniketa, tmgross, alkumar, gary, kjaju, kwankhede,
Danilo Krummrich, ojeda, targupta, lossin, bjorn3_gh, nova-gpu,
alex.gaynor, simona, jhubbard, aliceryhl
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Unchecked subtractions during framebuffer layout calculation can underflow if the device's video memory size is unexpectedly small.
--
--- Patch [6]: [PATCH v5 6/6] gpu: nova-core: reserve vGPU WPR2 heap ---
commit f23fa75b7168b0143fdda0d52c83938a849ca02d
Author: Zhi Wang <zhiw@nvidia.com>
gpu: nova-core: reserve vGPU WPR2 heap
This commit updates the framebuffer layout calculation to pass the detected
vGPU state. It keeps baremetal boots on the existing heap sizing path and
uses a firmware-dependent 570.144 vGPU default heap binding when vGPU is
enabled.
> diff --git a/drivers/gpu/nova-core/fb.rs b/drivers/gpu/nova-core/fb.rs
> --- a/drivers/gpu/nova-core/fb.rs
> +++ b/drivers/gpu/nova-core/fb.rs
[ ... ]
> @@ -234,10 +240,19 @@ pub(crate) fn new(chipset: Chipset, bar: Bar0<'_>, gsp_fw: &GspFirmware) -> Resu
> FbRange(elf_addr..elf_addr + elf_size)
> };
>
> + let (vf_partition_count, wpr2_heap_size) = match vgpu_state {
> + VgpuState::Disabled => (
> + 0,
> + gsp::LibosParams::from_chipset(chipset).wpr_heap_size(chipset, fb.end)?,
> + ),
> + VgpuState::Enabled { total_vfs } => (
> + u8::try_from(total_vfs.get()).map_err(|_| EINVAL)?,
> + gsp::LibosParams::vgpu_wpr_heap_size(),
> + ),
> + };
> +
> let wpr2_heap = {
> const WPR2_HEAP_DOWN_ALIGN: Alignment = Alignment::new::<SZ_1M>();
> - let wpr2_heap_size =
> - gsp::LibosParams::from_chipset(chipset).wpr_heap_size(chipset, fb.end)?;
> let wpr2_heap_addr = (elf.start - wpr2_heap_size).align_down(WPR2_HEAP_DOWN_ALIGN);
[Severity: High]
This is a pre-existing issue, but could this subtraction underflow if the
hardware or a hypervisor reports a malformed, undersized framebuffer?
Since this patch introduces a hardcoded ~581MB requirement for
wpr2_heap_size in vGPU mode, the risk seems significantly amplified. If the
hardware-reported video memory size isn't large enough to contain the
bootloader, elf, and this new large heap, elf.start - wpr2_heap_size will
either panic or wrap around to a massive out-of-bounds address.
Should there be a validation check against the total fb_size before
allocating these ranges backwards from the end of the framebuffer?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260722073913.1807677-1-zhiw@nvidia.com?part=6
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v5 5/6] gpu: nova-core: set RMSetSriovMode for vGPU
2026-07-22 7:39 ` [PATCH v5 5/6] gpu: nova-core: set RMSetSriovMode for vGPU Zhi Wang
@ 2026-07-22 7:52 ` sashiko-bot
0 siblings, 0 replies; 9+ messages in thread
From: sashiko-bot @ 2026-07-22 7:52 UTC (permalink / raw)
To: dakr, acourbot, Zhi Wang
Cc: linux-kernel, kjaju, nova-gpu, simona, alkumar, zhiwang, smitra,
aniketa, boqun.feng, Danilo Krummrich, Alexandre Courbot, cjia,
gary, lossin, targupta, jhubbard, ankita, apopple, tmgross,
kwankhede, alex.gaynor, bjorn3_gh, ecourtney, joelagnelf, airlied,
a.hindborg, ojeda, aliceryhl
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] The `size` field in the `PACKED_REGISTRY_TABLE` header is initialized incorrectly, causing the string data offsets to exceed the advertised table size.
--
--- Patch [5]: [PATCH v5 5/6] gpu: nova-core: set RMSetSriovMode for vGPU ---
commit 7a8618e853f5431c98f463e1cf9330f35e524af1
Author: Zhi Wang <zhiw@nvidia.com>
gpu: nova-core: set RMSetSriovMode for vGPU
The GSP registry setup needs to advertise SR-IOV mode when nova-core boots
GSP for an enabled vGPU configuration. Without the registry entry, GSP-RM
is not told to initialize in the mode required by NVIDIA vGPU.
> diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
> index 08380de39048a..134f34b191744 100644
> --- a/drivers/gpu/nova-core/gsp/commands.rs
> +++ b/drivers/gpu/nova-core/gsp/commands.rs
[ ... ]
> @@ -104,6 +105,17 @@ pub(crate) fn new() -> Result<Self> {
> GFP_KERNEL,
> )?;
>
> + if matches!(vgpu_state, VgpuState::Enabled { .. }) {
> + // RMSetSriovMode - required when vGPU is enabled.
> + entries.push(
> + RegistryEntry {
> + key: "RMSetSriovMode",
> + value: 1,
> + },
> + GFP_KERNEL,
> + )?;
> + }
> +
> Ok(Self { entries })
> }
> }
[Severity: High]
This isn't a bug introduced by this patch, but the size field in the
PACKED_REGISTRY_TABLE appears to be initialized incorrectly, which might cause
the newly added RMSetSriovMode entry to silently fail or cause out-of-bounds
reads.
Looking at SetRegistry::init() in this same file:
drivers/gpu/nova-core/gsp/commands.rs:SetRegistry::init() {
fn init(&self) -> impl Init<Self::Command, Self::InitError> {
Self::Command::init(
self.entries.len() as u32,
self.variable_payload_len() as u32,
)
}
}
Does this initialize the size using only the payload length, omitting the
8-byte table header itself?
Since the string offsets calculated in init_variable_payload() correctly
include the 8-byte header size, won't the end of the string data exceed the
advertised table size by exactly 8 bytes?
Could this cause the GSP firmware to truncate the registry entries or perform
an out-of-bounds read?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260722073913.1807677-1-zhiw@nvidia.com?part=5
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2026-07-22 7:52 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-22 7:39 [PATCH v5 0/6] gpu: nova-core: boot GSP with vGPU enabled Zhi Wang
2026-07-22 7:39 ` [PATCH v5 1/6] PCI/IOV: Return unsigned int from pci_sriov_get_totalvfs() Zhi Wang
2026-07-22 7:39 ` [PATCH v5 2/6] rust: pci: add sriov_get_totalvfs() helper Zhi Wang
2026-07-22 7:39 ` [PATCH v5 3/6] gpu: nova-core: read vGPU mode from FSP via PRC protocol Zhi Wang
2026-07-22 7:39 ` [PATCH v5 4/6] gpu: nova-core: detect and store vGPU state Zhi Wang
2026-07-22 7:39 ` [PATCH v5 5/6] gpu: nova-core: set RMSetSriovMode for vGPU Zhi Wang
2026-07-22 7:52 ` sashiko-bot
2026-07-22 7:39 ` [PATCH v5 6/6] gpu: nova-core: reserve vGPU WPR2 heap Zhi Wang
2026-07-22 7:52 ` sashiko-bot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox