* [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver
@ 2026-09-05 8:11 Zhi Wang
2026-09-05 8:11 ` [PATCH 01/13] gpu: nova-core: vgpu: add post-GSP-boot vGPU initialization Zhi Wang
` (12 more replies)
0 siblings, 13 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang, kvm
Add the nova-core support needed to create and manage NVIDIA vGPU
instances, together with the NVIDIA vGPU VFIO variant driver that consumes
the lifecycle interface.
The control path introduced by this series is:
+-----------------------+ +-----------------------+
| Linux guest | | Windows guest |
| NVIDIA guest driver | | NVIDIA guest driver |
+-----------+-----------+ +-----------+-----------+
| |
+-----------v-----------+ +-----------v-----------+
| QEMU (VFIO user) | | QEMU (VFIO user) |
+-----------+-----------+ +-----------+-----------+
+----------------+----------------+
|
+-------v-------+
| VFIO core |
+-------+-------+
|
+---------------------v---------------------+
| NVIDIA vGPU VFIO variant driver |
+---------------------+---------------------+
|
+---------------+----------------------+
| binds PCI VF | PF lifecycle API
| | open / close / reset
v v
+--------+--------+ +-------------+-------------+
| PCI VF(s) | | nova-core (PF driver) |
| (virtual GPU) | | vGPU Manager |
+--------+--------+ | VRAM / channels ... |
| +-------------+-------------+
| |
+-------------v--------------------------------------v----------------+
| NVIDIA physical GPU |
| VF resources PF / GSP / VRAM |
+---------------------------------------------------------------------+
On the nova-core side, the PF driver owns the GPU firmware-facing part
of vGPU management. It already detects vGPU mode before GSP boot and
supplies the SR-IOV topology to the GPU System Processor firmware
(GSP-RM). This series extends it to:
- Retain the firmware-reported VMMU segment size and FIFO engine
ordering after GSP_INIT, together with nova-core's channel capacity.
- Add VramBlock and VramRegion ownership plus Bar1Map for bounded CPU
mappings of VRAM-backed control structures.
- Query GSP-RM through its command queue for the vGPU type already
assigned to a VF and its properties.
- Build a layout-validated, profile-wide pool of paired framebuffer and
management-heap slots, reserve contiguous channel ranges, and keep
live allocations in a per-PF instance registry.
- Use the typed r000 firmware bindings for control and response layouts,
message IDs, and firmware-defined region sizes.
- Handle GMC transactions over the GSP queues, match GMC replies by
command and sequence, dispatch interleaved RM RPCs from the shared
GSP-to-CPU message queue, and consume stale GMC messages.
- Implement the GMCAPI bootload, shutdown, and cleanup operations with
typed channel and resource maps for each plugin instance.
- Establish the BAR1-backed PluginRpc channel, negotiate its protocol,
and send the VM configuration and plugin BME-state update.
- Use per-instance CeUtils channels to scrub guest framebuffer memory
on allocation, reset, and shutdown. If ownership or completion is
uncertain, keep the affected VRAM and channels out of the allocators.
- Expose the three per-instance plugin log buffers through debugfs with
the header needed by nvlog_decoder.
- Keep resource locking, teardown, and rollback inside nova-core.
- Select the firmware-defined 48-VM WPR2 heap when a device advertises
more than 32 VFs.
On the VFIO side, the NVIDIA vGPU VFIO variant driver:
- Binds only to an NVIDIA VF explicitly selected through
driver_override.
- Derives the guest function ID (GFID) from the SR-IOV VF index and
registers a vfio-pci-core device.
- Presents the vGPU PCI device and subsystem IDs returned by nova-core,
and limits the reported framebuffer BAR size.
- Delegates the remaining PCI and VFIO operations to vfio-pci-core and
uses the standard VFIO physical-device helpers for IOMMUFD.
The ownership boundary between the two drivers is visible at these
lifecycle points:
- Open: after vfio_pci_core_enable(), the variant driver passes the PF
returned by pci_physfn(vf), the GFID, the VF's PCI
domain/bus/device/function (DBDF), and the calling process TGID to
nvidia_vgpu_open(). nova-core validates the PF and GFID, creates and
activates the instance, and returns the guest-visible PCI IDs and
BAR1 size. On failure, the variant driver disables the vfio-pci-core
device; on success, it calls
vfio_pci_core_finish_enable().
- Reset: the variant driver first calls nvidia_vgpu_reset(). nova-core
sends the plugin reset RPC and scrubs the guest framebuffer. Only
after that succeeds is VFIO_DEVICE_RESET passed to vfio-pci-core for
the PCI function reset.
- Close: the variant driver invokes the void nvidia_vgpu_close()
interface to request plugin shutdown, framebuffer scrubbing, and
resource release from nova-core, then closes the vfio-pci-core
device.
The cross-module interface is therefore limited to three GPL-only
symbols in NOVA_CORE_VGPU: nvidia_vgpu_open(), nvidia_vgpu_close(), and
nvidia_vgpu_reset().
This is a ground-up rework of the earlier vGPU RFC [1]. In particular,
the vGPU manager has been moved from the VFIO driver into the nova-core
implementation, and the VFIO side no longer carries copies of RM firmware
headers. The VFIO driver attaches to the VF and uses its PF's nova-core
state for management, as discussed in that thread.
The patches are organized as follows:
1-4 Initialize VgpuManager and add the memory and firmware building
blocks.
5-8 Allocate instances and implement GMC, plugin boot, and RPC.
9-10 Scrub guest framebuffer memory and expose plugin diagnostics.
11-12 Export the lifecycle API and add its VFIO consumer.
13 Select the larger WPR2 heap for 48-VF devices.
The series is based on the nova-core r000 GSP, memory-management,
SR-IOV, Rust bitmap/id-pool, and PCI abstraction work. The exact base
commit is recorded below, and the complete prerequisite stack is
available in [2].
[1] https://lore.kernel.org/kvm/20250903221111.3866249-1-zhiw@nvidia.com/
[2] https://github.com/zhiwang-nvidia/nova-core/tree/zhi/nova-vgpu-wip-nova-gsp-20260902
Alok Kumar (1):
gpu: nova-core: vgpu: add VRAM slot allocator
Zhi Wang (12):
gpu: nova-core: vgpu: add post-GSP-boot vGPU initialization
gpu: nova-core: mm: add VramBlock and Bar1Map
gpu: nova-core: vgpu: add r000 plugin bindings
gpu: nova-core: vgpu: add instance create/destroy
gpu: nova-core: gsp: add GMC transaction helpers
gpu: nova-core: vgpu: add vGPU bootload
gpu: nova-core: vgpu: implement PluginRpc channel and config params
gpu: nova-core: vgpu: scrub guest framebuffer memory with CeUtils
gpu: nova-core: vgpu: export plugin log buffers via debugfs
gpu: nova-core: vgpu: export lifecycle operations to VFIO
vfio/nvidia-vgpu: add the NVIDIA vGPU VFIO variant driver
gpu: nova-core: reserve the 48-VM WPR2 heap
drivers/gpu/nova-core/driver.rs | 42 +-
drivers/gpu/nova-core/fb.rs | 2 +-
drivers/gpu/nova-core/gpu.rs | 184 +++--
drivers/gpu/nova-core/gpu/channel.rs | 3 +
drivers/gpu/nova-core/gsp.rs | 29 +-
drivers/gpu/nova-core/gsp/boot.rs | 13 +-
drivers/gpu/nova-core/gsp/cmdq.rs | 356 +++++++--
drivers/gpu/nova-core/gsp/commands.rs | 78 +-
drivers/gpu/nova-core/gsp/fw.rs | 63 +-
drivers/gpu/nova-core/gsp/fw/commands.rs | 185 ++++-
.../gpu/nova-core/gsp/fw/r000_00/bindings.rs | 175 +++++
drivers/gpu/nova-core/gsp/hal/gh100.rs | 2 +-
drivers/gpu/nova-core/gsp/hal/tu102.rs | 2 +-
drivers/gpu/nova-core/mm.rs | 6 +-
drivers/gpu/nova-core/mm/bar_user.rs | 183 ++++-
drivers/gpu/nova-core/mm/vram.rs | 187 +++++
drivers/gpu/nova-core/nova_core_exports.c | 5 +
drivers/gpu/nova-core/vgpu.rs | 91 ---
drivers/gpu/nova-core/vgpu/bootload.rs | 162 ++++
drivers/gpu/nova-core/vgpu/consts.rs | 33 +
drivers/gpu/nova-core/vgpu/fw.rs | 490 ++++++++++++
drivers/gpu/nova-core/vgpu/fw/commands.rs | 28 +
drivers/gpu/nova-core/vgpu/instance.rs | 705 ++++++++++++++++++
drivers/gpu/nova-core/vgpu/log.rs | 167 +++++
drivers/gpu/nova-core/vgpu/mod.rs | 170 +++++
drivers/gpu/nova-core/vgpu/plugin_rpc.rs | 274 +++++++
drivers/gpu/nova-core/vgpu/scrubber.rs | 474 ++++++++++++
drivers/gpu/nova-core/vgpu/vfio.rs | 282 +++++++
drivers/gpu/nova-core/vgpu/vram.rs | 140 ++++
drivers/vfio/pci/Kconfig | 2 +
drivers/vfio/pci/Makefile | 2 +
drivers/vfio/pci/nvidia-vgpu/Kconfig | 16 +
drivers/vfio/pci/nvidia-vgpu/Makefile | 2 +
drivers/vfio/pci/nvidia-vgpu/main.c | 253 +++++++
include/drm/nvidia_vgpu.h | 28 +
rust/bindings/bindings_helper.h | 1 +
36 files changed, 4536 insertions(+), 299 deletions(-)
create mode 100644 drivers/gpu/nova-core/mm/vram.rs
delete mode 100644 drivers/gpu/nova-core/vgpu.rs
create mode 100644 drivers/gpu/nova-core/vgpu/bootload.rs
create mode 100644 drivers/gpu/nova-core/vgpu/consts.rs
create mode 100644 drivers/gpu/nova-core/vgpu/fw.rs
create mode 100644 drivers/gpu/nova-core/vgpu/fw/commands.rs
create mode 100644 drivers/gpu/nova-core/vgpu/instance.rs
create mode 100644 drivers/gpu/nova-core/vgpu/log.rs
create mode 100644 drivers/gpu/nova-core/vgpu/mod.rs
create mode 100644 drivers/gpu/nova-core/vgpu/plugin_rpc.rs
create mode 100644 drivers/gpu/nova-core/vgpu/scrubber.rs
create mode 100644 drivers/gpu/nova-core/vgpu/vfio.rs
create mode 100644 drivers/gpu/nova-core/vgpu/vram.rs
create mode 100644 drivers/vfio/pci/nvidia-vgpu/Kconfig
create mode 100644 drivers/vfio/pci/nvidia-vgpu/Makefile
create mode 100644 drivers/vfio/pci/nvidia-vgpu/main.c
create mode 100644 include/drm/nvidia_vgpu.h
base-commit: a1dc9fdea9ab15e5df1bd6af1f6de2e40079f02d
--
2.53.0
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 01/13] gpu: nova-core: vgpu: add post-GSP-boot vGPU initialization
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 02/13] gpu: nova-core: mm: add VramBlock and Bar1Map Zhi Wang
` (11 subsequent siblings)
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
GSP-RM does not expose the parameters needed to divide resources among
vGPU instances until GSP_INIT completes. Before this point VgpuManager
only knows whether vGPU mode is enabled, so it cannot provide the engine
topology, VMMU alignment, or channel capacity required by instance
management.
Decode the VMMU segment size and ordered FIFO engine table from the typed
GSP_INIT NVKV response. Retain only host-driven engines while preserving
hardware FIFO order. After a successful GSP_INIT, initialize the manager
with these values and the 2048-channel capacity, but retain them only when
vGPU mode is enabled.
Move VgpuManager into Gpu and let it borrow the pinned ChannelIdPool.
Create that pool with the same channel capacity, order the Gpu fields so
the manager is dropped before the memory manager, GSP resources, and its
channel pool, and pass it separately to GSP boot so unload resources do
not retain a manager reference. Keep a copy of the detected mode in the
GPU-owned GSP runtime data instead of passing it through the HAL calls.
Co-developed-by: Alok Kumar <alkumar@nvidia.com>
Signed-off-by: Alok Kumar <alkumar@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/gpu.rs | 53 ++++++---
drivers/gpu/nova-core/gpu/channel.rs | 3 +
drivers/gpu/nova-core/gsp.rs | 12 +-
drivers/gpu/nova-core/gsp/boot.rs | 13 ++-
drivers/gpu/nova-core/gsp/commands.rs | 33 ++++++
drivers/gpu/nova-core/gsp/fw/commands.rs | 36 ++++++
drivers/gpu/nova-core/gsp/hal/gh100.rs | 2 +-
drivers/gpu/nova-core/gsp/hal/tu102.rs | 2 +-
drivers/gpu/nova-core/vgpu.rs | 91 ---------------
drivers/gpu/nova-core/vgpu/mod.rs | 139 +++++++++++++++++++++++
10 files changed, 274 insertions(+), 110 deletions(-)
delete mode 100644 drivers/gpu/nova-core/vgpu.rs
create mode 100644 drivers/gpu/nova-core/vgpu/mod.rs
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index 2105e993ee60..a2189589422a 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -53,6 +53,11 @@
mod channel;
mod hal;
+pub(crate) use self::channel::{
+ ChannelIdPool,
+ TOTAL_CHANNELS, //
+};
+
macro_rules! define_chipset {
({ $($variant:ident = $value:expr),* $(,)* }) =>
{
@@ -291,8 +296,6 @@ struct GspResources<'gpu> {
// TODO: use different resource types for each boot method, and make the relevant Gsp methods
// generic against them.
fsp: Option<Fsp<'gpu>>,
- /// vGPU state detected before GSP boot.
- vgpu: VgpuManager,
/// GSP runtime data.
#[pin]
gsp: Gsp,
@@ -304,6 +307,10 @@ struct GspResources<'gpu> {
#[pin_data]
pub(crate) struct Gpu<'gpu> {
spec: Spec,
+ /// vGPU state and firmware parameters.
+ ///
+ /// Declared before MM and BAR1 so live instances are torn down before their resources.
+ vgpu: VgpuManager<'gpu>,
/// GPU memory manager owning memory management resources.
///
/// Must be kept declared *before* `gsp_resources`, so that its components are dropped while
@@ -314,6 +321,11 @@ pub(crate) struct Gpu<'gpu> {
/// GSP and its resources.
#[pin]
gsp_resources: GspResources<'gpu>,
+ /// Channel ID pool borrowed by the vGPU manager and its live instances.
+ ///
+ /// Declared after `vgpu` so the manager is dropped before the pool.
+ #[pin]
+ chid_pool: ChannelIdPool,
/// System memory page required for flushing all pending GPU-side memory writes done through
/// PCIE into system memory, via sysmembar (A GPU-initiated HW memory-barrier operation).
///
@@ -342,7 +354,6 @@ fn drop(self: Pin<&mut Self>) {
gsp_falcon: &*this.gsp_falcon,
sec2_falcon: &*this.sec2_falcon,
fsp: this.fsp.as_mut(),
- vgpu: &*this.vgpu,
},
bundle,
)
@@ -402,6 +413,18 @@ pub(crate) fn new(
// Initialize this early because `gsp_resources` depends on it.
sysmem_flush: SysmemFlush::register(dev, bar, spec.chipset)?,
+ chid_pool <- ChannelIdPool::new(cv!(TOTAL_CHANNELS as usize)),
+
+ // TODO: Use `&chid_pool` self-referential pin-init syntax once available.
+ //
+ // SAFETY: `chid_pool` is initialized before this expression and lives at a pinned
+ // stable address. Field order drops `vgpu` before `chid_pool`, including unwind of
+ // an incomplete initializer.
+ vgpu: VgpuManager::new(
+ // SAFETY: The lifetime and drop-order rationale above covers this borrow.
+ unsafe { &*core::ptr::from_ref(chid_pool.as_ref().get_ref()) },
+ ),
+
gsp_resources <- try_pin_init!(GspResources {
device: pdev,
@@ -420,22 +443,26 @@ pub(crate) fn new(
fsp: Fsp::try_new(dev, bar, spec.chipset)?,
- vgpu: VgpuManager::new(pdev, spec.chipset, fsp.as_mut()),
+ _: {
+ vgpu.detect_state(pdev, spec.chipset, fsp.as_mut());
+ },
- gsp <- Gsp::new(pdev, spec.chipset),
+ gsp <- Gsp::new(pdev, spec.chipset, vgpu.state()),
// This member must be initialized last, so the unload bundle can never be dropped
// from outside of the constructed `GspResources`, ensuring that the unload sequence
// is properly run in case of failure.
- boot_result: gsp.boot(GspBootContext {
- pdev,
- bar,
- chipset: spec.chipset,
- gsp_falcon,
- sec2_falcon,
- fsp: fsp.as_mut(),
+ boot_result: gsp.boot(
+ GspBootContext {
+ pdev,
+ bar,
+ chipset: spec.chipset,
+ gsp_falcon,
+ sec2_falcon,
+ fsp: fsp.as_mut(),
+ },
vgpu,
- })?,
+ )?,
}),
_: {
diff --git a/drivers/gpu/nova-core/gpu/channel.rs b/drivers/gpu/nova-core/gpu/channel.rs
index 485efaba059d..9bf2edddca03 100644
--- a/drivers/gpu/nova-core/gpu/channel.rs
+++ b/drivers/gpu/nova-core/gpu/channel.rs
@@ -21,6 +21,9 @@
}, //
};
+/// Total channel ID capacity available to vGPU instances.
+pub(crate) const TOTAL_CHANNELS: u32 = 2048;
+
/// Pool for tracking reservations of channel IDs.
#[pin_data]
pub(crate) struct ChannelIdPool {
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index 700842240c22..deaae033a88f 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -60,7 +60,7 @@
fw::GspArgumentsPadded, //
},
num,
- vgpu::VgpuManager, //
+ vgpu::VgpuState, //
};
pub(crate) const GSP_PAGE_SHIFT: usize = 12;
@@ -79,7 +79,6 @@ pub(crate) struct GspBootContext<'ctx, 'gpu> {
pub(crate) gsp_falcon: &'ctx Falcon<'gpu, GspFalcon>,
pub(crate) sec2_falcon: &'ctx Falcon<'gpu, Sec2Falcon>,
pub(crate) fsp: Option<&'ctx mut Fsp<'gpu>>,
- pub(crate) vgpu: &'ctx VgpuManager,
}
impl<'ctx, 'gpu> GspBootContext<'ctx, 'gpu> {
@@ -282,6 +281,8 @@ struct LogBuffers {
pub(crate) struct Gsp {
/// Preloaded GSP firmware TLV metadata used during boot.
gsp_tlv: kernel::firmware::Firmware,
+ /// vGPU mode detected before GSP boot.
+ vgpu_state: VgpuState,
/// Libos arguments.
pub(crate) libos: Coherent<[LibosMemoryRegionInitArgument]>,
/// Log buffers for all LIBOS3 tasks, exposed via debugfs.
@@ -300,6 +301,7 @@ impl Gsp {
pub(crate) fn new(
pdev: &pci::Device<device::Bound>,
chipset: Chipset,
+ vgpu_state: VgpuState,
) -> impl PinInit<Self, Error> + '_ {
pin_init::pin_init_scope(move || {
let dev = pdev.as_ref();
@@ -323,6 +325,7 @@ pub(crate) fn new(
Ok(try_pin_init!(Self {
gsp_tlv,
+ vgpu_state,
cmdq: Arc::pin_init(Cmdq::new(dev), GFP_KERNEL)?,
rm_state_monitor: Coherent::zeroed(dev, GFP_KERNEL)?,
rmargs: Coherent::init(
@@ -398,6 +401,11 @@ pub(crate) fn new(
})
}
+ /// Returns the vGPU mode detected for this boot.
+ pub(crate) const fn vgpu_state(&self) -> VgpuState {
+ self.vgpu_state
+ }
+
/// Returns a shared handle to the GSP command queue.
pub(crate) fn cmdq(&self) -> Arc<Cmdq> {
self.cmdq.clone()
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 9c4d25f609de..2d027d2a9a35 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -37,6 +37,7 @@
gsp::GspFirmware,
radix3::Radix3, //
},
+ gpu::TOTAL_CHANNELS,
gsp::{
cmdq::{
Cmdq,
@@ -52,7 +53,8 @@
}, //
},
num,
- regs, //
+ regs,
+ vgpu::VgpuManager, //
};
impl super::Gsp {
@@ -72,6 +74,7 @@ impl super::Gsp {
pub(crate) fn boot(
self: Pin<&mut Self>,
mut ctx: super::GspBootContext<'_, '_>,
+ vgpu: &mut VgpuManager<'_>,
) -> Result<super::BootResult> {
let pdev = ctx.pdev;
let bar = ctx.bar;
@@ -128,7 +131,7 @@ pub(crate) fn boot(
// the registry keys ride inside that one request. Its reply is also what says GSP-RM has
// finished starting, and the load-and-execute events it raises first are dispatched as
// they arrive.
- let init_payload = commands::build_gsp_init_payload(pdev, chipset, ctx.vgpu.state())?;
+ let init_payload = commands::build_gsp_init_payload(pdev, chipset, self.vgpu_state())?;
// Only the chipsets that raise `GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER` are shipped a
// `gen_bootloader.tlv`, so requesting it elsewhere fails the whole boot with `ENOENT`.
let bootloader = if super::hal::uses_generic_bootloader(chipset) {
@@ -155,6 +158,12 @@ pub(crate) fn boot(
)
})?;
+ vgpu.init(
+ &static_info.fifo_engine_list,
+ static_info.vmmu_segment_size,
+ TOTAL_CHANNELS,
+ );
+
Ok(super::BootResult::new(
unload_guard.dismiss().1,
static_info,
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 02bb9d653253..89369ff23b85 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -49,6 +49,19 @@
vgpu::VgpuState, //
};
+/// Upper bound on entries in the hardware FIFO engine table.
+pub(crate) const MAX_FIFO_ENGINES: usize = 64;
+
+/// Bit mask for `NVGMC_SC_ENGINE_FLAGS_IS_HOST_DRIVEN` (bits 0:0).
+const ENGINE_FLAGS_IS_HOST_DRIVEN: u32 = 1 << 0;
+
+/// Ordered list of host-driven GMC engine IDs from the hardware FIFO engine table.
+#[derive(Copy, Clone)]
+pub(crate) struct FifoEngineList {
+ pub(crate) gmc_ids: [u32; MAX_FIFO_ENGINES],
+ pub(crate) count: usize,
+}
+
/// The static GPU configuration, as decoded from the `GSP_INIT` reply.
pub(crate) struct GetGspStaticInfoReply {
gpu_name: [u8; 64],
@@ -58,6 +71,10 @@ pub(crate) struct GetGspStaticInfoReply {
pub(crate) usable_fb_regions: KVec<Range<u64>>,
/// Exclusive end of the FB physical address space.
pub(crate) total_fb_end: u64,
+ /// VMMU segment size reported by GSP-RM, in bytes.
+ pub(crate) vmmu_segment_size: u64,
+ /// Ordered host-driven FIFO engine GMC IDs.
+ pub(crate) fifo_engine_list: FifoEngineList,
}
/// Error type for [`GetGspStaticInfoReply::gpu_name`].
@@ -289,12 +306,28 @@ fn decode_gsp_info(words: &[u64]) -> Result<GetGspStaticInfoReply> {
usable_fb_regions.push(region, GFP_KERNEL)?;
}
let total_fb_end = decoded.total_fb_end().ok_or(EINVAL)?;
+ let vmmu_segment_size = decoded.vmmu_segment_size();
+ let fifo_count = decoded.fifo_engine_count();
+ let raw_ids = decoded.fifo_engine_gmc_ids();
+ let raw_flags = decoded.fifo_engine_flags();
+ let mut fifo_engine_list = FifoEngineList {
+ gmc_ids: [0; MAX_FIFO_ENGINES],
+ count: 0,
+ };
+ for index in 0..fifo_count {
+ if raw_flags[index] & ENGINE_FLAGS_IS_HOST_DRIVEN != 0 {
+ fifo_engine_list.gmc_ids[fifo_engine_list.count] = raw_ids[index];
+ fifo_engine_list.count += 1;
+ }
+ }
Ok(GetGspStaticInfoReply {
gpu_name,
bar1_pde_base: decoded.bar1_pde_base(),
usable_fb_regions,
total_fb_end,
+ vmmu_segment_size,
+ fifo_engine_list,
})
}
diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs
index 22e4546142f0..1610b005199f 100644
--- a/drivers/gpu/nova-core/gsp/fw/commands.rs
+++ b/drivers/gpu/nova-core/gsp/fw/commands.rs
@@ -26,6 +26,7 @@
Encodeable,
Encoder,
Index,
+ Indexed,
Key,
KeyId,
Required, //
@@ -295,6 +296,14 @@ pub(crate) struct GspInitResponseSchema => GspInitResponse {
fb_regions: Accumulated<FbRegionSchema>,
bar1_pde_base: Required<u64, { Self::BAR1_PDE_BASE_KEY }>,
vmmu_segment_size: Key<u64, { Self::VMMU_SEGMENT_SIZE_KEY }>,
+ fifo_engine_count: Key<u32, { Self::FIFO_ENGINE_COUNT_KEY }>,
+ fifo_engine_gmc_ids: Indexed<
+ u32,
+ { GspInitResponse::MAX_FIFO_ENGINES },
+ { Self::FIFO_ENGINE_GMC_ENGINE_ID_KEY },
+ >,
+ fifo_engine_flags:
+ Indexed<u32, { GspInitResponse::MAX_FIFO_ENGINES }, { Self::FIFO_ENGINE_FLAGS_KEY }>,
}
}
@@ -303,6 +312,9 @@ impl GspInitResponseSchema {
const GPU_NAME_STRING_KEY: KeyId = 0x2000;
const BAR1_PDE_BASE_KEY: KeyId = 0x1020;
const VMMU_SEGMENT_SIZE_KEY: KeyId = 0x1050;
+ const FIFO_ENGINE_COUNT_KEY: KeyId = 0x0500;
+ const FIFO_ENGINE_GMC_ENGINE_ID_KEY: KeyId = 0x0501;
+ const FIFO_ENGINE_FLAGS_KEY: KeyId = 0x0502;
}
/// Payload of the `GSP_INIT` response.
@@ -312,10 +324,14 @@ pub(crate) struct GspInitResponse {
fb_regions: KVVec<FbRegion>,
bar1_pde_base: u64,
vmmu_segment_size: u64,
+ fifo_engine_count: u32,
+ fifo_engine_gmc_ids: [u32; Self::MAX_FIFO_ENGINES],
+ fifo_engine_flags: [u32; Self::MAX_FIFO_ENGINES],
}
impl GspInitResponse {
pub(crate) const MAX_GPU_NAME_LEN: usize = 64;
+ const MAX_FIFO_ENGINES: usize = 64;
/// A region with no tag is general-purpose memory. A tagged region is reserved for a
/// firmware-internal use that the tag identifies.
@@ -358,6 +374,26 @@ pub(crate) fn total_fb_end(&self) -> Option<u64> {
.max()?
.checked_add(1)
}
+
+ /// Returns the VMMU segment size reported by GSP-RM.
+ pub(crate) const fn vmmu_segment_size(&self) -> u64 {
+ self.vmmu_segment_size
+ }
+
+ /// Returns the count of FIFO engines reported by GSP-RM.
+ pub(crate) fn fifo_engine_count(&self) -> usize {
+ (self.fifo_engine_count as usize).min(Self::MAX_FIFO_ENGINES)
+ }
+
+ /// Returns the raw array of GMC engine IDs from the FIFO engine table.
+ pub(crate) fn fifo_engine_gmc_ids(&self) -> &[u32; Self::MAX_FIFO_ENGINES] {
+ &self.fifo_engine_gmc_ids
+ }
+
+ /// Returns the raw array of per-engine flags from the FIFO engine table.
+ pub(crate) fn fifo_engine_flags(&self) -> &[u32; Self::MAX_FIFO_ENGINES] {
+ &self.fifo_engine_flags
+ }
}
nvkv_decode! {
diff --git a/drivers/gpu/nova-core/gsp/hal/gh100.rs b/drivers/gpu/nova-core/gsp/hal/gh100.rs
index e283429a95dd..41c18d623b7a 100644
--- a/drivers/gpu/nova-core/gsp/hal/gh100.rs
+++ b/drivers/gpu/nova-core/gsp/hal/gh100.rs
@@ -151,7 +151,7 @@ fn boot(
let chipset = ctx.chipset;
let gsp_falcon = ctx.gsp_falcon;
- let fb_sizes = FbSizes::new(chipset, ctx.bar, ctx.vgpu.state())?;
+ let fb_sizes = FbSizes::new(chipset, ctx.bar, gsp.vgpu_state())?;
dev_dbg!(dev, "{:#x?}\n", fb_sizes);
let wpr_meta =
diff --git a/drivers/gpu/nova-core/gsp/hal/tu102.rs b/drivers/gpu/nova-core/gsp/hal/tu102.rs
index 74ea172726cb..bd47787348c8 100644
--- a/drivers/gpu/nova-core/gsp/hal/tu102.rs
+++ b/drivers/gpu/nova-core/gsp/hal/tu102.rs
@@ -259,7 +259,7 @@ fn boot(
let gsp_falcon = ctx.gsp_falcon;
let sec2_falcon = ctx.sec2_falcon;
- let fb_ranges = FbRanges::new(chipset, bar, gsp_fw, ctx.vgpu.state())?;
+ let fb_ranges = FbRanges::new(chipset, bar, gsp_fw, gsp.vgpu_state())?;
dev_dbg!(dev, "{:#x?}\n", fb_ranges);
// Declared before the unload guard so that if Booter fails while running, SEC2 is reset
diff --git a/drivers/gpu/nova-core/vgpu.rs b/drivers/gpu/nova-core/vgpu.rs
deleted file mode 100644
index 6b7e045acea8..000000000000
--- a/drivers/gpu/nova-core/vgpu.rs
+++ /dev/null
@@ -1,91 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-use core::num::NonZero;
-
-use kernel::{
- device,
- pci,
- prelude::*, //
-};
-
-use crate::{
- fsp::{
- Fsp,
- VgpuMode, //
- },
- gpu::Chipset, //
-};
-
-mod hal;
-
-/// vGPU state detected during GPU construction.
-#[derive(Debug, Clone, Copy)]
-pub(crate) enum VgpuState {
- /// vGPU mode is not enabled for this boot.
- Disabled,
- /// vGPU mode is enabled for this boot.
- Enabled {
- /// Total number of SR-IOV VFs supported by this device.
- total_vfs: NonZero<u16>,
- },
-}
-
-/// vGPU state manager.
-pub(crate) struct VgpuManager {
- state: VgpuState,
-}
-
-impl VgpuManager {
- /// Creates a vGPU manager by querying SR-IOV and the FSP PRC vGPU knob.
- pub(crate) fn new(
- pdev: &pci::Device<device::Core<'_>>,
- chipset: Chipset,
- fsp: Option<&mut Fsp<'_>>,
- ) -> Self {
- let state = Self::detect_state(pdev, chipset, fsp).unwrap_or_else(|e| {
- dev_warn!(
- pdev,
- "vGPU state detection failed: {:?}; disabling vGPU\n",
- e
- );
- VgpuState::Disabled
- });
- dev_dbg!(pdev, "vGPU state: {:?}\n", state);
-
- Self { state }
- }
-
- /// Detects the vGPU state from the chipset, SR-IOV capability and FSP PRC knob.
- fn detect_state(
- pdev: &pci::Device<device::Core<'_>>,
- chipset: Chipset,
- fsp: Option<&mut Fsp<'_>>,
- ) -> Result<VgpuState> {
- if !hal::vgpu_hal(chipset).supports_vgpu() {
- return Ok(VgpuState::Disabled);
- }
-
- let Some(total_vfs) = pdev.sriov_get_totalvfs() else {
- return Ok(VgpuState::Disabled);
- };
-
- if total_vfs.get() < 2 {
- // The current vGPU path does not support single-VF SR-IOV devices yet.
- // Treat one total VF as vGPU-disabled for now; single-VF support can relax
- // this gate once the manager handles that topology.
- return Ok(VgpuState::Disabled);
- }
-
- let fsp = fsp.ok_or(ENODEV)?;
-
- match fsp.read_vgpu_mode(pdev.as_ref())? {
- VgpuMode::Enabled => Ok(VgpuState::Enabled { total_vfs }),
- VgpuMode::Disabled => Ok(VgpuState::Disabled),
- }
- }
-
- /// Returns the detected vGPU state for this boot.
- pub(crate) fn state(&self) -> VgpuState {
- self.state
- }
-}
diff --git a/drivers/gpu/nova-core/vgpu/mod.rs b/drivers/gpu/nova-core/vgpu/mod.rs
new file mode 100644
index 000000000000..1354c662a507
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/mod.rs
@@ -0,0 +1,139 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+use core::num::NonZero;
+
+use kernel::{
+ device,
+ pci,
+ prelude::*, //
+};
+
+use crate::{
+ fsp::{
+ Fsp,
+ VgpuMode, //
+ },
+ gpu::{
+ ChannelIdPool,
+ Chipset, //
+ },
+ gsp::commands::FifoEngineList, //
+};
+
+mod hal;
+
+/// vGPU state detected during GPU construction.
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum VgpuState {
+ /// vGPU mode is not enabled for this boot.
+ Disabled,
+ /// vGPU mode is enabled for this boot.
+ Enabled {
+ /// Total number of SR-IOV VFs supported by this device.
+ total_vfs: NonZero<u16>,
+ },
+}
+
+/// vGPU state manager.
+pub(crate) struct VgpuManager<'gpu> {
+ /// Channel ID pool the per-VF areas are reserved from.
+ #[expect(dead_code)]
+ pub(crate) chid_pool: &'gpu ChannelIdPool,
+ state: VgpuState,
+ vmmu_segment_size: Option<u64>,
+ total_channels: Option<u32>,
+ fifo_engine_list: Option<FifoEngineList>,
+}
+
+impl<'gpu> VgpuManager<'gpu> {
+ /// Creates an empty vGPU manager for initialization during GPU construction.
+ pub(crate) const fn new(chid_pool: &'gpu ChannelIdPool) -> Self {
+ Self {
+ chid_pool,
+ state: VgpuState::Disabled,
+ vmmu_segment_size: None,
+ total_channels: None,
+ fifo_engine_list: None,
+ }
+ }
+
+ /// Detects and stores vGPU state before GSP boot.
+ pub(crate) fn detect_state(
+ &mut self,
+ pdev: &pci::Device<device::Core<'_>>,
+ chipset: Chipset,
+ fsp: Option<&mut Fsp<'_>>,
+ ) {
+ let state: Result<VgpuState> = (|| {
+ if !hal::vgpu_hal(chipset).supports_vgpu() {
+ return Ok(VgpuState::Disabled);
+ }
+
+ let Some(total_vfs) = pdev.sriov_get_totalvfs() else {
+ return Ok(VgpuState::Disabled);
+ };
+
+ if total_vfs.get() < 2 {
+ // The current vGPU path does not support single-VF SR-IOV devices yet.
+ // Treat one total VF as vGPU-disabled for now; single-VF support can relax
+ // this gate once the manager handles that topology.
+ return Ok(VgpuState::Disabled);
+ }
+
+ let fsp = fsp.ok_or(ENODEV)?;
+
+ match fsp.read_vgpu_mode(pdev.as_ref())? {
+ VgpuMode::Enabled => Ok(VgpuState::Enabled { total_vfs }),
+ VgpuMode::Disabled => Ok(VgpuState::Disabled),
+ }
+ })();
+
+ self.state = state.unwrap_or_else(|e| {
+ dev_warn!(
+ pdev,
+ "vGPU state detection failed: {:?}; disabling vGPU\n",
+ e
+ );
+ VgpuState::Disabled
+ });
+ dev_dbg!(pdev, "vGPU state: {:?}\n", self.state);
+ }
+
+ /// Returns the detected vGPU state for this boot.
+ pub(crate) fn state(&self) -> VgpuState {
+ self.state
+ }
+
+ /// Initializes the runtime parameters returned by GSP_INIT.
+ pub(crate) fn init(
+ &mut self,
+ fifo_engine_list: &FifoEngineList,
+ vmmu_segment_size: u64,
+ total_channels: u32,
+ ) {
+ if matches!(self.state, VgpuState::Enabled { .. }) {
+ self.vmmu_segment_size = Some(vmmu_segment_size);
+ self.total_channels = Some(total_channels);
+ self.fifo_engine_list = Some(*fifo_engine_list);
+ }
+ }
+
+ /// Returns the firmware-reported VMMU segment size when vGPU is enabled.
+ #[expect(dead_code)]
+ pub(crate) const fn vmmu_segment_size(&self) -> Option<u64> {
+ self.vmmu_segment_size
+ }
+
+ /// Returns the number of channel IDs available to vGPU instances.
+ #[expect(dead_code)]
+ pub(crate) const fn total_channels(&self) -> Option<u32> {
+ self.total_channels
+ }
+
+ /// Returns the ordered FIFO engine list provided by GSP_INIT.
+ #[expect(dead_code)]
+ pub(crate) fn fifo_engine_list(&self) -> Result<&FifoEngineList> {
+ self.fifo_engine_list.as_ref().ok_or(ENODEV)
+ }
+}
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 02/13] gpu: nova-core: mm: add VramBlock and Bar1Map
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
2026-09-05 8:11 ` [PATCH 01/13] gpu: nova-core: vgpu: add post-GSP-boot vGPU initialization Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 03/13] gpu: nova-core: vgpu: add VRAM slot allocator Zhi Wang
` (10 subsequent siblings)
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
GPU page table setup and VRAM-backed control structures require the
driver to allocate physical VRAM and map it into the BAR1 aperture for
CPU access. These operations are common to both the base driver and
vGPU paths.
VramBlock owns a buddy allocator allocation. Shared VramRegion views
keep that allocation alive while callers select byte ranges within a
larger preallocated block. Bar1Map retains one such region while mapping
the containing pages and bounds all CPU accesses to the requested view.
The mapping must be explicitly destroyed to release GPU VA resources and
invalidate PTEs.
Keep BarUser inline in Gpu and let short-lived BarUserAccess objects
borrow it. Bar1Map owns its mapped VA range and borrows the driver-owned
BAR1 mapping; explicit destruction returns the VA through BarUser and
GpuMm. Its MMIO accessors remain runtime checked because BAR1 and the
logical mapping have runtime sizes.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/gpu.rs | 20 ++-
drivers/gpu/nova-core/mm.rs | 6 +-
drivers/gpu/nova-core/mm/bar_user.rs | 143 ++++++++++++++++++--
drivers/gpu/nova-core/mm/vram.rs | 187 +++++++++++++++++++++++++++
4 files changed, 329 insertions(+), 27 deletions(-)
create mode 100644 drivers/gpu/nova-core/mm/vram.rs
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index a2189589422a..430d0cc12546 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -317,7 +317,8 @@ pub(crate) struct Gpu<'gpu> {
/// the GSP is still operational.
mm: GpuMm<'gpu>,
/// BAR1 user interface for CPU access to GPU virtual memory.
- bar_user: Arc<BarUser<'gpu>>,
+ #[pin]
+ bar_user: BarUser<'gpu>,
/// GSP and its resources.
#[pin]
gsp_resources: GspResources<'gpu>,
@@ -508,19 +509,16 @@ pub(crate) fn new(
},
// Create BAR1 user interface for CPU access to GPU virtual memory.
- bar_user: {
+ bar_user <- {
let info = &gsp_resources.boot_result.static_info;
let pdb_addr = VramAddress::from_raw(info.bar1_pde_base);
let bar1_idx = crate::driver::bar1_resource_index(pdev)?;
let bar1_size = pdev.resource_len(bar1_idx)?;
- Arc::pin_init(
- BarUser::new(
- pdb_addr,
- gsp_resources.spec.chipset,
- bar1_size,
- bar1,
- )?,
- GFP_KERNEL,
+ BarUser::new(
+ pdb_addr,
+ gsp_resources.spec.chipset,
+ bar1_size,
+ bar1,
)?
},
})
@@ -543,7 +541,7 @@ pub(crate) fn run_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bou
dev,
this.mm,
regions,
- this.bar_user,
+ this.bar_user.as_ref().get_ref(),
info.bar1_pde_base,
this.spec.chipset,
) {
diff --git a/drivers/gpu/nova-core/mm.rs b/drivers/gpu/nova-core/mm.rs
index a5bc4042577b..d1cad5ad52fc 100644
--- a/drivers/gpu/nova-core/mm.rs
+++ b/drivers/gpu/nova-core/mm.rs
@@ -67,6 +67,7 @@ macro_rules! impl_pfn_bounded {
mod regs;
pub(super) mod tlb;
pub(super) mod vmm;
+pub(crate) mod vram;
/// GPU Memory Manager - owns all core MM components.
///
@@ -298,8 +299,7 @@ pub(crate) mod selftest {
use kernel::{
device,
- sizes::SizeConstants,
- sync::Arc, //
+ sizes::SizeConstants, //
};
use super::*;
@@ -309,7 +309,7 @@ pub(crate) fn run(
dev: &device::Device<device::Bound>,
mm: &mut GpuMm<'_>,
usable_fb_regions: &[Range<u64>],
- bar_user: &Arc<bar_user::BarUser<'_>>,
+ bar_user: &bar_user::BarUser<'_>,
bar1_pdb: u64,
chipset: Chipset,
) -> Result {
diff --git a/drivers/gpu/nova-core/mm/bar_user.rs b/drivers/gpu/nova-core/mm/bar_user.rs
index fb2129c47f47..adc23ac6d467 100644
--- a/drivers/gpu/nova-core/mm/bar_user.rs
+++ b/drivers/gpu/nova-core/mm/bar_user.rs
@@ -7,15 +7,13 @@
io::Io,
new_mutex,
prelude::*,
- sync::{
- Arc,
- Mutex, //
- },
+ sync::Mutex, //
};
use crate::{
driver::Bar1,
gpu::Chipset,
+ mm::vram::VramRegion,
mm::{
vmm::{
MappedRange,
@@ -60,12 +58,12 @@ pub(crate) fn new(
}
/// Map physical pages to a contiguous BAR1 virtual range.
- pub(crate) fn map(
- self: &Arc<Self>,
+ pub(crate) fn map<'access>(
+ &'access self,
mm: &mut GpuMm<'_>,
pfns: &[Pfn],
writable: bool,
- ) -> Result<BarUserAccess<'gpu>> {
+ ) -> Result<BarUserAccess<'access, 'gpu>> {
if pfns.is_empty() {
return Err(EINVAL);
}
@@ -73,22 +71,22 @@ pub(crate) fn map(
let mapped = vmm.map_pages(mm, pfns, None, writable)?;
Ok(BarUserAccess {
- bar_user: self.clone(),
+ bar_user: self,
mapped: Some(mapped),
})
}
}
/// Access object for a mapped BAR1 region.
-pub(crate) struct BarUserAccess<'gpu> {
- bar_user: Arc<BarUser<'gpu>>,
+pub(crate) struct BarUserAccess<'access, 'gpu> {
+ bar_user: &'access BarUser<'gpu>,
/// [`BarUserAccess::release`] [`Option::take`]s this; `Some` at
/// drop time means `release()` was never called.
mapped: Option<MappedRange>,
}
#[expect(dead_code)]
-impl BarUserAccess<'_> {
+impl BarUserAccess<'_, '_> {
/// Tear down the BAR1 mapping.
pub(crate) fn release(mut self, mm: &mut GpuMm<'_>) -> Result {
let mapped = self.mapped.take().ok_or(EINVAL)?;
@@ -162,7 +160,7 @@ pub(crate) fn try_write64(&self, value: u64, offset: usize) -> Result {
}
}
-impl Drop for BarUserAccess<'_> {
+impl Drop for BarUserAccess<'_, '_> {
fn drop(&mut self) {
if self.mapped.is_some() {
kernel::pr_warn!(
@@ -174,6 +172,125 @@ fn drop(&mut self) {
}
}
+/// An owned BAR1 mapping of a region within a live VRAM allocation.
+///
+/// The mapping retains the region's backing allocation until its PTEs have been removed. A
+/// logical region may begin or end within a page; the containing pages are mapped while CPU
+/// access remains bounded to the requested byte range.
+pub(crate) struct Bar1Map<'gpu> {
+ bar1: &'gpu Bar1<'gpu>,
+ mapped: MappedRange,
+ region: VramRegion,
+ page_bias: usize,
+ logical_size: usize,
+}
+
+impl<'gpu> Bar1Map<'gpu> {
+ /// Maps a VRAM region through BAR1.
+ pub(crate) fn new(
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ region: VramRegion,
+ writable: bool,
+ ) -> Result<Self> {
+ let page_size = u64::try_from(PAGE_SIZE).map_err(|_| EOVERFLOW)?;
+ let region_start = region.address();
+ let region_end = region_start.checked_add(region.size()).ok_or(EOVERFLOW)?;
+ let map_start = region_start - region_start % page_size;
+ let map_end =
+ region_end.checked_add(page_size - 1).ok_or(EOVERFLOW)? / page_size * page_size;
+ let map_size = map_end.checked_sub(map_start).ok_or(EINVAL)?;
+ let num_pages = usize::try_from(map_size / page_size).map_err(|_| EOVERFLOW)?;
+ if num_pages == 0 {
+ return Err(EINVAL);
+ }
+
+ let page_bias = usize::try_from(region_start - map_start).map_err(|_| EOVERFLOW)?;
+ let logical_size = usize::try_from(region.size()).map_err(|_| EOVERFLOW)?;
+ let mut pfns = KVec::new();
+ for page in 0..num_pages {
+ let byte_offset = u64::try_from(page)
+ .map_err(|_| EOVERFLOW)?
+ .checked_mul(page_size)
+ .ok_or(EOVERFLOW)?;
+ let address = map_start.checked_add(byte_offset).ok_or(EOVERFLOW)?;
+ pfns.push(Pfn::from(VramAddress::from_raw(address)), GFP_KERNEL)?;
+ }
+
+ let mut vmm = bar_user.vmm.lock();
+ let mapped = vmm.map_pages(mm, &pfns, None, writable)?;
+
+ Ok(Self {
+ bar1: bar_user.bar1,
+ mapped,
+ region,
+ page_bias,
+ logical_size,
+ })
+ }
+
+ /// Returns the mapped physical VRAM region.
+ pub(crate) fn region(&self) -> &VramRegion {
+ &self.region
+ }
+
+ /// Returns the logical GPU virtual address visible through BAR1.
+ pub(crate) fn gpu_va_addr(&self) -> Result<u64> {
+ VirtualAddress::from(self.mapped.vfn_start)
+ .into_raw()
+ .checked_add(u64::try_from(self.page_bias).map_err(|_| EOVERFLOW)?)
+ .ok_or(EOVERFLOW)
+ }
+
+ /// Returns the requested logical mapping size.
+ pub(crate) const fn size(&self) -> usize {
+ self.logical_size
+ }
+
+ fn bar_offset(&self, offset: usize, width: usize) -> Result<usize> {
+ let logical_end = offset.checked_add(width).ok_or(EOVERFLOW)?;
+ if logical_end > self.logical_size {
+ return Err(EINVAL);
+ }
+
+ let access_offset = self.page_bias.checked_add(offset).ok_or(EOVERFLOW)?;
+ if !access_offset.is_multiple_of(width) {
+ return Err(EINVAL);
+ }
+
+ let base_vfn: usize = self.mapped.vfn_start.raw().into_safe_cast();
+ let base = base_vfn.checked_mul(PAGE_SIZE).ok_or(EOVERFLOW)?;
+ base.checked_add(access_offset).ok_or(EOVERFLOW)
+ }
+
+ // BAR1 and the logical mapping have runtime sizes, so these accessors
+ // validate the offset, width, and alignment before performing MMIO.
+ pub(crate) fn try_read32(&self, offset: usize) -> Result<u32> {
+ self.bar1
+ .try_read32(self.bar_offset(offset, size_of::<u32>())?)
+ }
+
+ pub(crate) fn try_write32(&self, value: u32, offset: usize) -> Result {
+ self.bar1
+ .try_write32(value, self.bar_offset(offset, size_of::<u32>())?)
+ }
+
+ pub(crate) fn try_write64(&self, value: u64, offset: usize) -> Result {
+ self.bar1
+ .try_write64(value, self.bar_offset(offset, size_of::<u64>())?)
+ }
+
+ /// Invalidates the PTEs and releases the BAR1 virtual address.
+ ///
+ /// The backing VRAM region remains alive until unmapping completes.
+ pub(crate) fn destroy(self, bar_user: &BarUser<'gpu>, mm: &mut GpuMm<'_>) -> Result {
+ let mut vmm = bar_user.vmm.lock();
+ let result = vmm.unmap_pages(mm, self.mapped);
+ drop(self.region);
+ result
+ }
+}
+
/// Run MM subsystem self-tests during probe.
///
/// Tests page table infrastructure and `BAR1` MMIO access using the `BAR1`
@@ -183,7 +300,7 @@ fn drop(&mut self) {
pub(crate) fn run_self_test(
dev: &device::Device<device::Bound>,
mm: &mut GpuMm<'_>,
- bar_user: &Arc<BarUser<'_>>,
+ bar_user: &BarUser<'_>,
bar1_pdb: u64,
chipset: Chipset,
) -> Result {
diff --git a/drivers/gpu/nova-core/mm/vram.rs b/drivers/gpu/nova-core/mm/vram.rs
new file mode 100644
index 000000000000..4a4bd42c9f18
--- /dev/null
+++ b/drivers/gpu/nova-core/mm/vram.rs
@@ -0,0 +1,187 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+//! VRAM allocation and allocation-relative region helpers.
+
+use core::ops::Range;
+
+use kernel::{
+ gpu::buddy::{
+ AllocatedBlocks,
+ GpuBuddyAllocFlags,
+ GpuBuddyAllocMode, //
+ },
+ prelude::*,
+ ptr::Alignment,
+ sync::Arc, //
+};
+
+use super::{
+ GpuMm,
+ PAGE_SIZE, //
+};
+
+/// A physically contiguous VRAM allocation shared by its regions.
+///
+/// The buddy allocation is returned only after the block and every region or
+/// BAR1 mapping backed by it have been dropped.
+pub(crate) struct VramBlock {
+ _blocks: Pin<KBox<AllocatedBlocks>>,
+ address: u64,
+ size: u64,
+}
+
+impl VramBlock {
+ /// Return the physical start address of the allocation.
+ pub(crate) const fn address(&self) -> u64 {
+ self.address
+ }
+
+ /// Return the allocation size in bytes.
+ pub(crate) const fn size(&self) -> u64 {
+ self.size
+ }
+
+ /// Create a checked allocation-relative region.
+ pub(crate) fn region(self: &Arc<Self>, range: Range<u64>) -> Result<VramRegion> {
+ VramRegion::new(self.clone(), range)
+ }
+
+ /// Create a region spanning the complete allocation.
+ pub(crate) fn full_region(self: &Arc<Self>) -> VramRegion {
+ VramRegion {
+ backing: self.clone(),
+ address: self.address,
+ size: self.size,
+ }
+ }
+}
+
+/// A byte range within a shared [`VramBlock`].
+#[derive(Clone)]
+pub(crate) struct VramRegion {
+ backing: Arc<VramBlock>,
+ address: u64,
+ size: u64,
+}
+
+impl VramRegion {
+ fn new(backing: Arc<VramBlock>, range: Range<u64>) -> Result<Self> {
+ let size = range
+ .end
+ .checked_sub(range.start)
+ .filter(|size| *size != 0)
+ .ok_or(EINVAL)?;
+ if range.end > backing.size {
+ return Err(EINVAL);
+ }
+ let address = backing.address.checked_add(range.start).ok_or(EOVERFLOW)?;
+ backing.address.checked_add(range.end).ok_or(EOVERFLOW)?;
+
+ Ok(Self {
+ backing,
+ address,
+ size,
+ })
+ }
+
+ /// Return the physical address of the first byte in this region.
+ pub(crate) const fn address(&self) -> u64 {
+ self.address
+ }
+
+ /// Return the region size in bytes.
+ pub(crate) const fn size(&self) -> u64 {
+ self.size
+ }
+
+ /// Return a checked subregion relative to this region.
+ pub(crate) fn subregion(&self, range: Range<u64>) -> Result<Self> {
+ let size = range
+ .end
+ .checked_sub(range.start)
+ .filter(|size| *size != 0)
+ .ok_or(EINVAL)?;
+ if range.end > self.size {
+ return Err(EINVAL);
+ }
+ let address = self.address.checked_add(range.start).ok_or(EOVERFLOW)?;
+ address.checked_add(size).ok_or(EOVERFLOW)?;
+
+ Ok(Self {
+ backing: self.backing.clone(),
+ address,
+ size,
+ })
+ }
+}
+
+/// Allocate an exact VRAM range relative to a usable region's buddy base.
+pub(crate) fn alloc_vram_range(
+ mm: &GpuMm<'_>,
+ range: Range<u64>,
+ align: u64,
+) -> Result<Arc<VramBlock>> {
+ let page_size = u64::try_from(PAGE_SIZE).map_err(|_| EOVERFLOW)?;
+ let size = range
+ .end
+ .checked_sub(range.start)
+ .filter(|size| *size != 0)
+ .ok_or(EINVAL)?;
+ if !range.start.is_multiple_of(page_size) || !size.is_multiple_of(page_size) {
+ return Err(EINVAL);
+ }
+
+ let align = align.max(page_size);
+ let align_usize = usize::try_from(align).map_err(|_| EOVERFLOW)?;
+ let min_block_size = Alignment::new_checked(align_usize).ok_or(EINVAL)?;
+ let buddy = mm.buddy();
+ if range.end > buddy.size() {
+ return Err(ENOSPC);
+ }
+
+ let blocks = KBox::pin_init(
+ buddy.alloc_blocks(
+ GpuBuddyAllocMode::Range(range.clone()),
+ size,
+ min_block_size,
+ GpuBuddyAllocFlags::default(),
+ ),
+ GFP_KERNEL,
+ )?;
+
+ let mut address = None;
+ let mut allocation_end = None;
+ let mut covered = 0u64;
+ for block in blocks.as_ref().iter() {
+ let block_address = block.offset();
+ let block_size = block.size();
+ let block_end = block_address.checked_add(block_size).ok_or(EOVERFLOW)?;
+ address = Some(address.map_or(block_address, |start: u64| start.min(block_address)));
+ allocation_end = Some(allocation_end.map_or(block_end, |end: u64| end.max(block_end)));
+ covered = covered.checked_add(block_size).ok_or(EOVERFLOW)?;
+ }
+
+ let address = address.ok_or(ENOMEM)?;
+ let allocation_end = allocation_end.ok_or(ENOMEM)?;
+ let expected_address = buddy
+ .base_offset()
+ .checked_add(range.start)
+ .ok_or(EOVERFLOW)?;
+ if address != expected_address
+ || covered != size
+ || allocation_end.checked_sub(address).ok_or(EIO)? != size
+ || !address.is_multiple_of(align)
+ {
+ return Err(EIO);
+ }
+
+ Ok(Arc::new(
+ VramBlock {
+ _blocks: blocks,
+ address,
+ size,
+ },
+ GFP_KERNEL,
+ )?)
+}
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 03/13] gpu: nova-core: vgpu: add VRAM slot allocator
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
2026-09-05 8:11 ` [PATCH 01/13] gpu: nova-core: vgpu: add post-GSP-boot vGPU initialization Zhi Wang
2026-09-05 8:11 ` [PATCH 02/13] gpu: nova-core: mm: add VramBlock and Bar1Map Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 04/13] gpu: nova-core: vgpu: add r000 plugin bindings Zhi Wang
` (9 subsequent siblings)
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
From: Alok Kumar <alkumar@nvidia.com>
vGPU profiles need fixed-size framebuffer and plugin management-heap
regions for each instance. Allocating them independently fragments VRAM
and does not preserve a stable profile-wide layout.
Add a standalone slot allocator that reserves one exact VRAM range and
tracks assignments in a bitmap. Validate the profile layout and reject
an allocation request whose layout differs from the active pool.
Lay out all framebuffer slots first and all management-heap slots
second, pairing them by bitmap index. Require the framebuffer stride and
pool base to satisfy the profile VMMU-segment alignment.
Keep the pool in an Arc<VramBlock> and return checked VramRegion views
for each slot. This lets consumers map a subrange while retaining the
complete buddy allocation for as long as any view remains alive.
Signed-off-by: Alok Kumar <alkumar@nvidia.com>
Co-developed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/vgpu/mod.rs | 1 +
drivers/gpu/nova-core/vgpu/vram.rs | 142 +++++++++++++++++++++++++++++
2 files changed, 143 insertions(+)
create mode 100644 drivers/gpu/nova-core/vgpu/vram.rs
diff --git a/drivers/gpu/nova-core/vgpu/mod.rs b/drivers/gpu/nova-core/vgpu/mod.rs
index 1354c662a507..a9d4860f18e3 100644
--- a/drivers/gpu/nova-core/vgpu/mod.rs
+++ b/drivers/gpu/nova-core/vgpu/mod.rs
@@ -22,6 +22,7 @@
};
mod hal;
+mod vram;
/// vGPU state detected during GPU construction.
#[derive(Debug, Clone, Copy)]
diff --git a/drivers/gpu/nova-core/vgpu/vram.rs b/drivers/gpu/nova-core/vgpu/vram.rs
new file mode 100644
index 000000000000..c646511b6fd6
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/vram.rs
@@ -0,0 +1,142 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+//! VRAM slot allocation for vGPU instances.
+
+#![expect(dead_code)]
+
+use kernel::{
+ bitmap::BitmapVec,
+ prelude::*,
+ sync::Arc, //
+};
+
+use crate::mm::{
+ vram::{
+ alloc_vram_range,
+ VramBlock,
+ VramRegion, //
+ },
+ GpuMm, //
+};
+
+const VRAM_SLOT_MIN_ALIGN: u64 = 4096;
+
+#[derive(Clone, Copy, PartialEq)]
+pub(crate) struct VgpuVramLayout {
+ pub type_id: u32,
+ pub max_slots: u32,
+ pub fb_size: u64,
+ pub heap_size: u64,
+ pub fb_align: u64,
+}
+
+impl VgpuVramLayout {
+ fn validated(mut self) -> Result<Self> {
+ if self.max_slots == 0 || self.fb_size == 0 || self.heap_size == 0 {
+ return Err(EINVAL);
+ }
+ self.fb_align = core::cmp::max(self.fb_align, VRAM_SLOT_MIN_ALIGN);
+ if !self.fb_align.is_power_of_two() {
+ return Err(EINVAL);
+ }
+ if self.fb_size & (self.fb_align - 1) != 0
+ || self.heap_size & (VRAM_SLOT_MIN_ALIGN - 1) != 0
+ {
+ return Err(EINVAL);
+ }
+ Ok(self)
+ }
+}
+
+pub(crate) struct VgpuVramSlot {
+ index: usize,
+ pub fbmem: VramRegion,
+ pub mgmt_heap: VramRegion,
+}
+
+impl VgpuVramSlot {
+ /// Return this slot's index in its profile-wide allocation pool.
+ pub(crate) const fn index(&self) -> usize {
+ self.index
+ }
+}
+
+pub(super) struct VgpuVramSlotAllocator {
+ backing: Arc<VramBlock>,
+ layout: VgpuVramLayout,
+ fb_region_size: u64,
+ used: BitmapVec,
+}
+
+impl VgpuVramSlotAllocator {
+ pub(super) fn new(mm: &GpuMm<'_>, layout: VgpuVramLayout) -> Result<Self> {
+ let layout = layout.validated()?;
+ let max_slots = u64::from(layout.max_slots);
+ // Keep framebuffer slots contiguous so each starts at `fb_align`;
+ // interleaving page-aligned heaps could misalign later slots.
+ let fb_region_size = layout.fb_size.checked_mul(max_slots).ok_or(EINVAL)?;
+ let heap_region_size = layout.heap_size.checked_mul(max_slots).ok_or(EINVAL)?;
+ let pool_size = fb_region_size.checked_add(heap_region_size).ok_or(EINVAL)?;
+
+ let used = BitmapVec::new(
+ usize::try_from(layout.max_slots).map_err(|_| EINVAL)?,
+ GFP_KERNEL,
+ )?;
+ let backing = alloc_vram_range(mm, 0..pool_size, VRAM_SLOT_MIN_ALIGN)?;
+ if !backing.address().is_multiple_of(layout.fb_align) {
+ return Err(EINVAL);
+ }
+
+ Ok(Self {
+ backing,
+ layout,
+ fb_region_size,
+ used,
+ })
+ }
+
+ pub(super) fn matches_layout(&self, layout: VgpuVramLayout) -> Result<bool> {
+ Ok(self.layout == layout.validated()?)
+ }
+
+ pub(super) fn alloc(&mut self, layout: VgpuVramLayout) -> Result<VgpuVramSlot> {
+ if !self.matches_layout(layout)? {
+ return Err(EBUSY);
+ }
+
+ let bitmap_index = self.used.next_zero_bit(0).ok_or(ENOSPC)?;
+ let slot = u64::try_from(bitmap_index).map_err(|_| EINVAL)?;
+ let fb_offset = self.layout.fb_size.checked_mul(slot).ok_or(EINVAL)?;
+ let fb_end = fb_offset.checked_add(self.layout.fb_size).ok_or(EINVAL)?;
+ let heap_offset = self
+ .fb_region_size
+ .checked_add(self.layout.heap_size.checked_mul(slot).ok_or(EINVAL)?)
+ .ok_or(EINVAL)?;
+ let heap_end = heap_offset
+ .checked_add(self.layout.heap_size)
+ .ok_or(EINVAL)?;
+ let fbmem = self.backing.region(fb_offset..fb_end)?;
+ let mgmt_heap = self.backing.region(heap_offset..heap_end)?;
+
+ self.used.set_bit(bitmap_index);
+
+ Ok(VgpuVramSlot {
+ index: bitmap_index,
+ fbmem,
+ mgmt_heap,
+ })
+ }
+
+ /// Drop a slot's region views and return its reservation to this pool.
+ pub(super) fn release(&mut self, slot: VgpuVramSlot) {
+ let index = slot.index;
+ debug_assert_eq!(self.used.next_bit(index), Some(index));
+ drop(slot);
+ self.used.clear_bit(index);
+ }
+
+ pub(super) fn is_empty(&self) -> bool {
+ self.used.last_bit().is_none()
+ }
+}
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 04/13] gpu: nova-core: vgpu: add r000 plugin bindings
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
` (2 preceding siblings ...)
2026-09-05 8:11 ` [PATCH 03/13] gpu: nova-core: vgpu: add VRAM slot allocator Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 05/13] gpu: nova-core: vgpu: add instance create/destroy Zhi Wang
` (8 subsequent siblings)
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
The vGPU plugin communication area and RPC protocol are defined by the
r000 firmware interface. Keeping local copies of that ABI risks letting
the host driver drift from the firmware layout.
Extend the existing r000 binding set with the nova-core subset of
dev_vgpu_gsp_shared.h. The bindings expose the control and response
region types, message IDs, and firmware-defined region sizes. Keep the
raw declarations private and expose only the symbols needed by the vGPU
plugin communication layer.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
.../gpu/nova-core/gsp/fw/r000_00/bindings.rs | 173 ++++++++++++++++++
drivers/gpu/nova-core/vgpu/fw.rs | 2 +
drivers/gpu/nova-core/vgpu/mod.rs | 1 +
3 files changed, 176 insertions(+)
create mode 100644 drivers/gpu/nova-core/vgpu/fw.rs
diff --git a/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs b/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
index 4861500b2747..dcb44a403469 100644
--- a/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
+++ b/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
@@ -1,4 +1,5 @@
// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#[repr(C)]
#[derive(Default)]
@@ -857,3 +858,175 @@ pub struct rpc_unloading_guest_driver_v1F_07 {
pub __bindgen_padding_0: [u8; 2usize],
pub newLevel: u32_,
}
+pub const GSP_PLUGIN_BOOTLOADED: u32 = 1315261039;
+pub const VGPU_CPU_GSP_CTRL_BUFF_VERSION: u32 = 2;
+pub const VGPU_CPU_GSP_CTRL_BUFF_REGION_SIZE: u32 = 4096;
+pub const VGPU_CPU_GSP_RESPONSE_BUFF_REGION_SIZE: u32 = 4096;
+pub const VGPU_CPU_GSP_MESSAGE_BUFF_REGION_SIZE: u32 = 4096;
+pub const VGPU_CPU_GSP_MIGRATION_BUFF_REGION_SIZE: u32 = 2097152;
+pub const VGPU_CPU_GSP_ERROR_BUFF_REGION_SIZE: u32 = 4096;
+pub const VGPU_CPU_GSP_INIT_TASK_LOG_BUFF_REGION_SIZE: u32 = 131072;
+pub const VGPU_CPU_GSP_VGPU_TASK_LOG_BUFF_REGION_SIZE: u32 = 262144;
+pub const VGPU_CPU_GSP_KERNEL_TASK_LOG_BUFF_REGION_SIZE: u32 = 65536;
+pub const VGPU_CPU_GSP_GUEST_RPC_TRACE_BUFF_REGION_SIZE: u32 = 65536;
+pub const VGPU_CPU_GSP_COMMUNICATION_BUFF_TOTAL_SIZE: u32 = 2637824;
+pub type VGPU_CPU_GSP_BOOL = u32_;
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct VGPU_CPU_GSP_VGX_VERSION {
+ pub major_number: u32_,
+ pub minor_number: u32_,
+}
+#[repr(C)]
+#[derive(Debug, Copy, Clone, MaybeZeroable)]
+pub struct VGPU_CPU_GSP_GUEST_INFO {
+ pub vgx_version: VGPU_CPU_GSP_VGX_VERSION,
+ pub guest_driver_version_buffer_length: u32_,
+ pub guest_version_buffer_length: u32_,
+ pub guest_title_buffer_length: u32_,
+ pub guest_changelist_number: u32_,
+ pub guest_driver_version_buffer: [ffi::c_char; 256usize],
+ pub guest_version_buffer: [ffi::c_char; 256usize],
+ pub guest_title_buffer: [ffi::c_char; 256usize],
+ pub guest_branch_buffer: [ffi::c_char; 256usize],
+}
+impl Default for VGPU_CPU_GSP_GUEST_INFO {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+#[repr(C)]
+#[derive(Copy, Clone, MaybeZeroable)]
+pub union VGPU_CPU_GSP_CTRL_BUFF_REGION {
+ pub buf: [u8_; 4096usize],
+ pub __bindgen_anon_1: VGPU_CPU_GSP_CTRL_BUFF_REGION__bindgen_ty_1,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct VGPU_CPU_GSP_CTRL_BUFF_REGION__bindgen_ty_1 {
+ pub version: u32_,
+ pub message_type: u32_,
+ pub message_seq_num: u32_,
+ pub __bindgen_padding_0: [u8; 4usize],
+ pub response_buff_offset: u64_,
+ pub message_buff_offset: u64_,
+ pub migration_buff_offset: u64_,
+ pub error_buff_offset: u64_,
+ pub guest_rpc_trace_buff_offset: u64_,
+ pub migration_buf_cpu_access_offset: u32_,
+ pub is_migration_in_progress: u8_,
+ pub __bindgen_padding_1: [u8; 3usize],
+ pub error_buff_cpu_get_idx: u32_,
+ pub guest_rpc_trace_buff_cpu_get_idx: u32_,
+ pub attached_vgpu_count: u32_,
+ pub is_gr_init_done: u8_,
+ pub __bindgen_padding_2: [u8; 3usize],
+ pub host_info: [VGPU_CPU_GSP_CTRL_BUFF_REGION__bindgen_ty_1__bindgen_ty_1; 16usize],
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct VGPU_CPU_GSP_CTRL_BUFF_REGION__bindgen_ty_1__bindgen_ty_1 {
+ pub vgpu_type_id: u32_,
+ pub host_gpu_pci_id: u32_,
+ pub pci_dev_id: u32_,
+ pub vgpu_uuid: [u8_; 16usize],
+}
+impl Default for VGPU_CPU_GSP_CTRL_BUFF_REGION {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_VERSION_NEGOTIATION: MESSAGE = 1;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_SETUP_CONFIG_PARAMS_AND_INIT: MESSAGE = 2;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_RESET: MESSAGE = 3;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_MIGRATION_STOP_WORK: MESSAGE = 4;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_MIGRATION_CANCEL_STOP: MESSAGE = 5;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_MIGRATION_SAVE_STATE: MESSAGE = 6;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_MIGRATION_CANCEL_SAVE: MESSAGE = 7;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_MIGRATION_RESTORE_STATE: MESSAGE = 8;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_MIGRATION_RESTORE_DEFERRED_STATE: MESSAGE = 9;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_MIGRATION_RESUME_WORK: MESSAGE = 10;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_CONSOLE_VNC_STATE: MESSAGE = 11;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_VF_BAR0_REG_ACCESS: MESSAGE = 12;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_UPDATE_BME_STATE: MESSAGE = 13;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_RESET_MIGRATION_BUFFER_PTR: MESSAGE = 14;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_RELEASE_CLIENT_DATABASE: MESSAGE = 15;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_CHECK_IS_ALIVE: MESSAGE = 16;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_SEND_STATIC_INFO: MESSAGE = 17;
+pub const MESSAGE_NV_VGPU_CPU_RPC_MSG_MAX: MESSAGE = 18;
+pub type MESSAGE = ffi::c_uint;
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct VGPU_CPU_GSP_DISPLAYLESS_SURFACE {
+ pub sequence_update_start: u64_,
+ pub sequence_update_end: u64_,
+ pub effective_fb_page_size: u32_,
+ pub rect_width: u32_,
+ pub rect_height: u32_,
+ pub surface_width: u32_,
+ pub surface_height: u32_,
+ pub surface_size: u32_,
+ pub surface_offset: u32_,
+ pub surface_format: u32_,
+ pub surface_kind: u32_,
+ pub surface_pitch: u32_,
+ pub surface_type: u32_,
+ pub surface_block_height: u8_,
+ pub __bindgen_padding_0: [u8; 3usize],
+ pub is_blanking_enabled: VGPU_CPU_GSP_BOOL,
+ pub is_flip_pending: VGPU_CPU_GSP_BOOL,
+ pub is_free_pending: VGPU_CPU_GSP_BOOL,
+ pub is_memory_blocklinear: VGPU_CPU_GSP_BOOL,
+}
+#[repr(C)]
+#[derive(Copy, Clone, MaybeZeroable)]
+pub union VGPU_CPU_GSP_RESPONSE_BUFF_REGION {
+ pub buf: [u8_; 4096usize],
+ pub __bindgen_anon_1: VGPU_CPU_GSP_RESPONSE_BUFF_REGION__bindgen_ty_1,
+}
+#[repr(C)]
+#[derive(Debug, Copy, Clone, MaybeZeroable)]
+pub struct VGPU_CPU_GSP_RESPONSE_BUFF_REGION__bindgen_ty_1 {
+ pub message_seq_num_received: u32_,
+ pub message_seq_num_processed: u32_,
+ pub result_code: u32_,
+ pub guest_rpc_version: u32_,
+ pub migration_buf_gsp_access_offset: u32_,
+ pub migration_state_save_complete: u32_,
+ pub is_migration_allowed: VGPU_CPU_GSP_BOOL,
+ pub __bindgen_padding_0: [u8; 4usize],
+ pub surface: [VGPU_CPU_GSP_DISPLAYLESS_SURFACE; 4usize],
+ pub error_buff_gsp_put_idx: u32_,
+ pub grid_license_state: u32_,
+ pub guest_os_type: u32_,
+ pub frl_config: u32_,
+ pub guest_info: VGPU_CPU_GSP_GUEST_INFO,
+ pub is_guest_info_populated: VGPU_CPU_GSP_BOOL,
+ pub guest_rpc_trace_buff_gsp_put_idx: u32_,
+}
+impl Default for VGPU_CPU_GSP_RESPONSE_BUFF_REGION__bindgen_ty_1 {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+impl Default for VGPU_CPU_GSP_RESPONSE_BUFF_REGION {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
diff --git a/drivers/gpu/nova-core/vgpu/fw.rs b/drivers/gpu/nova-core/vgpu/fw.rs
new file mode 100644
index 000000000000..edfb0f984b6d
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/fw.rs
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
diff --git a/drivers/gpu/nova-core/vgpu/mod.rs b/drivers/gpu/nova-core/vgpu/mod.rs
index a9d4860f18e3..a96d0018fa3d 100644
--- a/drivers/gpu/nova-core/vgpu/mod.rs
+++ b/drivers/gpu/nova-core/vgpu/mod.rs
@@ -21,6 +21,7 @@
gsp::commands::FifoEngineList, //
};
+mod fw;
mod hal;
mod vram;
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 05/13] gpu: nova-core: vgpu: add instance create/destroy
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
` (3 preceding siblings ...)
2026-09-05 8:11 ` [PATCH 04/13] gpu: nova-core: vgpu: add r000 plugin bindings Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 06/13] gpu: nova-core: gsp: add GMC transaction helpers Zhi Wang
` (7 subsequent siblings)
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
Add the instance registry and the resources needed for an individual
vGPU. Allocate paired framebuffer and management-heap regions from a
profile-wide VRAM slot pool, and reserve channel IDs through
ChannelIdPool.
Keep mandatory allocations as non-optional fields, enforce
profile-specific instance limits, and group the firmware identity into
an InstanceInfo. Add typed NVKV GMCAPI helpers for querying the VF
assignment and vGPU properties.
Make VgpuManager own the instance registry. Each registry entry keeps
the instance's channel reservation and VRAM regions allocated until the
entry is removed.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/gpu.rs | 72 +++---
drivers/gpu/nova-core/gsp.rs | 1 +
drivers/gpu/nova-core/gsp/boot.rs | 4 +-
drivers/gpu/nova-core/gsp/cmdq.rs | 63 +++++
drivers/gpu/nova-core/gsp/commands.rs | 11 +
drivers/gpu/nova-core/gsp/fw.rs | 8 +
drivers/gpu/nova-core/gsp/fw/commands.rs | 44 ++--
drivers/gpu/nova-core/mm/vram.rs | 3 +
drivers/gpu/nova-core/vgpu/consts.rs | 12 +
drivers/gpu/nova-core/vgpu/instance.rs | 288 +++++++++++++++++++++++
drivers/gpu/nova-core/vgpu/mod.rs | 48 ++--
11 files changed, 486 insertions(+), 68 deletions(-)
create mode 100644 drivers/gpu/nova-core/vgpu/consts.rs
create mode 100644 drivers/gpu/nova-core/vgpu/instance.rs
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index 430d0cc12546..5c12847c19bc 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -49,12 +49,12 @@
vgpu::VgpuManager, //
};
-#[cfg_attr(not(CONFIG_KUNIT = "y"), expect(dead_code))]
mod channel;
mod hal;
pub(crate) use self::channel::{
ChannelIdPool,
+ ChannelIdReservation,
TOTAL_CHANNELS, //
};
@@ -310,6 +310,7 @@ pub(crate) struct Gpu<'gpu> {
/// vGPU state and firmware parameters.
///
/// Declared before MM and BAR1 so live instances are torn down before their resources.
+ #[pin]
vgpu: VgpuManager<'gpu>,
/// GPU memory manager owning memory management resources.
///
@@ -421,50 +422,55 @@ pub(crate) fn new(
// SAFETY: `chid_pool` is initialized before this expression and lives at a pinned
// stable address. Field order drops `vgpu` before `chid_pool`, including unwind of
// an incomplete initializer.
- vgpu: VgpuManager::new(
+ vgpu <- VgpuManager::new(
// SAFETY: The lifetime and drop-order rationale above covers this borrow.
unsafe { &*core::ptr::from_ref(chid_pool.as_ref().get_ref()) },
),
- gsp_resources <- try_pin_init!(GspResources {
- device: pdev,
+ gsp_resources <- {
+ let mut vgpu = vgpu;
+ try_pin_init!(GspResources {
+ device: pdev,
- spec: *spec,
+ spec: *spec,
- bar,
+ bar,
- gsp_falcon: Falcon::new(
- dev,
- spec.chipset,
- bar
- )
- .inspect(|falcon| falcon.clear_swgen0_intr())?,
+ gsp_falcon: Falcon::new(
+ dev,
+ spec.chipset,
+ bar
+ )
+ .inspect(|falcon| falcon.clear_swgen0_intr())?,
- sec2_falcon: Falcon::new(dev, spec.chipset, bar)?,
+ sec2_falcon: Falcon::new(dev, spec.chipset, bar)?,
- fsp: Fsp::try_new(dev, bar, spec.chipset)?,
+ fsp: Fsp::try_new(dev, bar, spec.chipset)?,
- _: {
- vgpu.detect_state(pdev, spec.chipset, fsp.as_mut());
- },
+ _: {
+ vgpu.as_mut().detect_state(pdev, spec.chipset, fsp.as_mut());
+ },
- gsp <- Gsp::new(pdev, spec.chipset, vgpu.state()),
-
- // This member must be initialized last, so the unload bundle can never be dropped
- // from outside of the constructed `GspResources`, ensuring that the unload sequence
- // is properly run in case of failure.
- boot_result: gsp.boot(
- GspBootContext {
- pdev,
- bar,
- chipset: spec.chipset,
- gsp_falcon,
- sec2_falcon,
- fsp: fsp.as_mut(),
+ gsp <- Gsp::new(pdev, spec.chipset, vgpu.as_ref().state()),
+
+ // This member must be initialized last, so the unload bundle can never be
+ // dropped from outside of the constructed `GspResources`, ensuring that the
+ // unload sequence is properly run in case of failure.
+ boot_result: {
+ gsp.boot(
+ GspBootContext {
+ pdev,
+ bar,
+ chipset: spec.chipset,
+ gsp_falcon,
+ sec2_falcon,
+ fsp: fsp.as_mut(),
+ },
+ vgpu.as_mut(),
+ )?
},
- vgpu,
- )?,
- }),
+ })
+ },
_: {
// The `GSP_INIT` reply already carried this.
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index deaae033a88f..2521d7331996 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -32,6 +32,7 @@
mod regs;
pub(crate) use fw::{
+ vgpu_bindings,
GspFmcBootParams,
GspFwWprMeta,
LibosMemoryRegionInitArgument,
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 2d027d2a9a35..659f3783ed13 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -74,7 +74,7 @@ impl super::Gsp {
pub(crate) fn boot(
self: Pin<&mut Self>,
mut ctx: super::GspBootContext<'_, '_>,
- vgpu: &mut VgpuManager<'_>,
+ mut vgpu: Pin<&mut VgpuManager<'_>>,
) -> Result<super::BootResult> {
let pdev = ctx.pdev;
let bar = ctx.bar;
@@ -158,7 +158,7 @@ pub(crate) fn boot(
)
})?;
- vgpu.init(
+ vgpu.as_mut().init(
&static_info.fifo_engine_list,
static_info.vmmu_segment_size,
TOTAL_CHANNELS,
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index e6931f65b167..e472ec94691d 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -517,6 +517,14 @@ pub(crate) enum QueuePointers {
Reset,
}
+/// Response from a GMC API command.
+pub(crate) struct GmcResponse {
+ /// Response status (`NV_STATUS` code). Zero means success.
+ pub(crate) status: u32,
+ /// Response payload copied out of the message queue.
+ pub(crate) payload: KVec<u8>,
+}
+
/// GSP command queue.
///
/// Provides the ability to send commands and receive messages from the GSP using a shared memory
@@ -680,6 +688,61 @@ pub(crate) fn send_gmc_no_wait(
.send_gmc(bar, command_id, payload, max_response_size)
}
+ /// Sends a GMC API command and waits for its response.
+ ///
+ /// The queue stays locked for the complete transaction. A single deadline bounds all queue
+ /// elements observed while waiting.
+ pub(crate) fn send_gmc_and_receive(
+ &self,
+ bar: Bar0<'_>,
+ command_id: u32,
+ payload: &[u8],
+ max_response_size: u32,
+ ) -> Result<GmcResponse> {
+ let mut inner = self.inner.lock();
+ inner.send_gmc(bar, command_id, payload, max_response_size)?;
+
+ let deadline = Instant::<Monotonic>::now() + Self::RECEIVE_TIMEOUT;
+ loop {
+ let remaining = deadline - Instant::<Monotonic>::now();
+ if remaining.is_negative() {
+ return Err(ETIMEDOUT);
+ }
+
+ let response = inner.receive_gmc_and_dispatch(
+ bar,
+ remaining,
+ |received_command, status, payload_0, payload_1| {
+ if received_command != command_id {
+ return (None, QueuePointers::Unchanged);
+ }
+
+ let response = (|| {
+ let mut payload = KVec::with_capacity(
+ payload_0
+ .len()
+ .checked_add(payload_1.len())
+ .ok_or(EOVERFLOW)?,
+ GFP_KERNEL,
+ )?;
+ payload.extend_from_slice(payload_0, GFP_KERNEL)?;
+ payload.extend_from_slice(payload_1, GFP_KERNEL)?;
+ Ok(GmcResponse {
+ status,
+ payload,
+ })
+ })();
+
+ (Some(response), QueuePointers::Unchanged)
+ },
+ )?;
+
+ if let Some(response) = response {
+ return response;
+ }
+ }
+ }
+
/// Waits for an unsolicited GSP event of type `M`, dispatching any other event that arrives
/// first.
///
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 89369ff23b85..417df31988c2 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -49,6 +49,11 @@
vgpu::VgpuState, //
};
+pub(crate) use fw::commands::{
+ Dbdf,
+ VgpuProperties, //
+};
+
/// Upper bound on entries in the hardware FIFO engine table.
pub(crate) const MAX_FIFO_ENGINES: usize = 64;
@@ -281,6 +286,12 @@ fn nvkv_words(payload_0: &[u8], payload_1: &[u8]) -> Result<KVVec<u64>> {
Ok(out)
}
+/// Decodes a byte-oriented GMC vGPU-properties response with the typed NVKV schema.
+pub(crate) fn decode_vgpu_properties(payload: &[u8]) -> Result<KBox<VgpuProperties>> {
+ let words = nvkv_words(payload, &[])?;
+ VgpuProperties::decode(&words)
+}
+
/// Decodes the static GPU configuration from an NVKV stream.
///
/// # Errors
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index c2869c7fdf2a..f8f7f85d2df0 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -4,6 +4,14 @@
pub(crate) mod commands;
mod r000_00;
+/// Raw firmware declarations used by vGPU management.
+pub(crate) mod vgpu_bindings {
+ pub(crate) use super::r000_00::{
+ GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_ASSIGNED_VF_VGPU_TYPE,
+ GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_VGPU_PROPERTIES, //
+ };
+}
+
// Alias to avoid repeating the version number with every use.
use r000_00 as bindings;
diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs
index 1610b005199f..0603fbde172f 100644
--- a/drivers/gpu/nova-core/gsp/fw/commands.rs
+++ b/drivers/gpu/nova-core/gsp/fw/commands.rs
@@ -22,6 +22,7 @@
Accumulated,
Array,
ArrayVec,
+ Decoder,
DecoderValue,
Encodeable,
Encoder,
@@ -29,7 +30,8 @@
Indexed,
Key,
KeyId,
- Required, //
+ Required,
+ UnknownKeyPolicy, //
};
use super::bindings;
@@ -653,27 +655,33 @@ impl VgpuPropertiesSchema {
const FB_RESERVATION_KEY: KeyId = 0x310F;
}
-struct VgpuProperties {
- name: ArrayVec<u8, { Self::STRING_LEN }>,
- class: ArrayVec<u8, { Self::STRING_LEN }>,
- type_id: u32,
- bar1_length: u64,
- max_instance: u32,
- ecc: u32,
- profile_size: u64,
- max_fps: u32,
- num_heads: u32,
- max_res_x: u32,
- max_res_y: u32,
- dev_id: u32,
- subsystem_id: u32,
- fb_length: u64,
- gsp_heap_size: u64,
- fb_reservation: u64,
+pub(crate) struct VgpuProperties {
+ pub(crate) name: ArrayVec<u8, { Self::STRING_LEN }>,
+ pub(crate) class: ArrayVec<u8, { Self::STRING_LEN }>,
+ pub(crate) type_id: u32,
+ pub(crate) bar1_length: u64,
+ pub(crate) max_instance: u32,
+ pub(crate) ecc: u32,
+ pub(crate) profile_size: u64,
+ pub(crate) max_fps: u32,
+ pub(crate) num_heads: u32,
+ pub(crate) max_res_x: u32,
+ pub(crate) max_res_y: u32,
+ pub(crate) dev_id: u32,
+ pub(crate) subsystem_id: u32,
+ pub(crate) fb_length: u64,
+ pub(crate) gsp_heap_size: u64,
+ pub(crate) fb_reservation: u64,
}
impl VgpuProperties {
const STRING_LEN: usize = 64;
+
+ /// Decodes an NVKV response using the typed schema.
+ pub(crate) fn decode(words: &[u64]) -> Result<KBox<Self>> {
+ let decoder = Decoder::new(words, UnknownKeyPolicy::Ignore);
+ KBox::try_init(decoder.decode(VgpuPropertiesSchema::default())?, GFP_KERNEL)
+ }
}
// SETUP_CONFIG_PARAMS_AND_INIT
diff --git a/drivers/gpu/nova-core/mm/vram.rs b/drivers/gpu/nova-core/mm/vram.rs
index 4a4bd42c9f18..87b7ce7f2c93 100644
--- a/drivers/gpu/nova-core/mm/vram.rs
+++ b/drivers/gpu/nova-core/mm/vram.rs
@@ -86,16 +86,19 @@ fn new(backing: Arc<VramBlock>, range: Range<u64>) -> Result<Self> {
}
/// Return the physical address of the first byte in this region.
+ #[expect(dead_code)]
pub(crate) const fn address(&self) -> u64 {
self.address
}
/// Return the region size in bytes.
+ #[expect(dead_code)]
pub(crate) const fn size(&self) -> u64 {
self.size
}
/// Return a checked subregion relative to this region.
+ #[expect(dead_code)]
pub(crate) fn subregion(&self, range: Range<u64>) -> Result<Self> {
let size = range
.end
diff --git a/drivers/gpu/nova-core/vgpu/consts.rs b/drivers/gpu/nova-core/vgpu/consts.rs
new file mode 100644
index 000000000000..7ec577ec12f2
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/consts.rs
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+/// Development OpenRM GMC command identifiers used by vGPU management.
+pub(crate) mod gmc {
+ use crate::gsp::vgpu_bindings as bindings;
+
+ pub(crate) const VGPU_MGMT_QUERY_PROPERTIES: u32 =
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_VGPU_PROPERTIES;
+ pub(crate) const VGPU_MGMT_QUERY_ASSIGNED_VF: u32 =
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_ASSIGNED_VF_VGPU_TYPE;
+}
diff --git a/drivers/gpu/nova-core/vgpu/instance.rs b/drivers/gpu/nova-core/vgpu/instance.rs
new file mode 100644
index 000000000000..ed304945330a
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/instance.rs
@@ -0,0 +1,288 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+use core::num::NonZeroUsize;
+
+use kernel::{
+ prelude::*,
+ ptr::Alignment,
+ sizes::SizeConstants, //
+};
+
+use crate::{
+ driver::Bar0,
+ gpu::ChannelIdReservation,
+ gsp::{
+ cmdq::Cmdq,
+ commands::{
+ decode_vgpu_properties,
+ Dbdf,
+ VgpuProperties, //
+ },
+ },
+ mm::GpuMm,
+ vgpu::{
+ consts::gmc,
+ vram::{
+ VgpuVramLayout,
+ VgpuVramSlot,
+ VgpuVramSlotAllocator, //
+ },
+ VgpuManager, //
+ },
+};
+
+/// Guest Function ID. GFID 0 is reserved for the PF; VFs start at 1.
+#[repr(transparent)]
+#[derive(Clone, Copy, PartialEq, Eq)]
+pub(crate) struct Gfid(pub(crate) u32);
+
+/// vGPU type descriptor populated from a typed NVKV properties response.
+#[expect(dead_code)]
+pub(crate) struct VgpuType {
+ name: [u8; 64],
+ class: [u8; 64],
+ vgpu_type_id: u32,
+ bar1_length: u64,
+ max_instance: u32,
+ ecc_supported: u32,
+ profile_size: u64,
+ max_fps: u32,
+ num_heads: u32,
+ max_res_x: u32,
+ max_res_y: u32,
+ pci_dev_id: u32,
+ pci_subsys_id: u32,
+ fb_length: u64,
+ gsp_heap_size: u64,
+ fb_reservation: u64,
+}
+
+impl VgpuType {
+ fn from_properties(properties: &VgpuProperties) -> Self {
+ let mut name = [0; 64];
+ let name_len = properties.name.len().min(name.len());
+ name[..name_len].copy_from_slice(&properties.name[..name_len]);
+
+ let mut class = [0; 64];
+ let class_len = properties.class.len().min(class.len());
+ class[..class_len].copy_from_slice(&properties.class[..class_len]);
+
+ Self {
+ name,
+ class,
+ vgpu_type_id: properties.type_id,
+ bar1_length: properties.bar1_length,
+ max_instance: properties.max_instance,
+ ecc_supported: properties.ecc,
+ profile_size: properties.profile_size,
+ max_fps: properties.max_fps,
+ num_heads: properties.num_heads,
+ max_res_x: properties.max_res_x,
+ max_res_y: properties.max_res_y,
+ pci_dev_id: properties.dev_id,
+ pci_subsys_id: properties.subsystem_id,
+ fb_length: properties.fb_length,
+ gsp_heap_size: properties.gsp_heap_size,
+ fb_reservation: properties.fb_reservation,
+ }
+ }
+}
+
+/// A vGPU instance and the resources reserved for it.
+#[expect(dead_code)]
+pub(crate) struct VgpuInstance<'gpu> {
+ pub(crate) gfid: Gfid,
+ pub(crate) dbdf: Dbdf,
+ pub(crate) vgpu_type: VgpuType,
+ pub(crate) vm_pid: u32,
+ pub(crate) chids: ChannelIdReservation<'gpu>,
+ pub(crate) num_plugin_channels: u32,
+ pub(crate) vram_slot: VgpuVramSlot,
+}
+
+/// Identity and firmware profile used to allocate an instance.
+pub(crate) struct InstanceInfo {
+ pub(crate) gfid: Gfid,
+ pub(crate) dbdf: Dbdf,
+ pub(crate) vgpu_type: VgpuType,
+ pub(crate) vm_pid: u32,
+}
+
+#[expect(dead_code)]
+impl InstanceInfo {
+ pub(crate) const fn new(gfid: Gfid, dbdf: Dbdf, vgpu_type: VgpuType, vm_pid: u32) -> Self {
+ Self {
+ gfid,
+ dbdf,
+ vgpu_type,
+ vm_pid,
+ }
+ }
+}
+
+/// Registry of live vGPU instances.
+pub(crate) struct VgpuInstances<'gpu> {
+ /// Declared before `vram_slots` so instance regions are dropped before their backing pool.
+ instances: KVec<VgpuInstance<'gpu>>,
+ vram_slots: Option<VgpuVramSlotAllocator>,
+}
+
+#[expect(dead_code)]
+impl<'gpu> VgpuInstances<'gpu> {
+ pub(crate) const fn new() -> Self {
+ Self {
+ instances: KVec::new(),
+ vram_slots: None,
+ }
+ }
+
+ fn alloc_vram_slot(&mut self, mm: &GpuMm<'_>, layout: VgpuVramLayout) -> Result<VgpuVramSlot> {
+ let replace_empty_pool = match self.vram_slots.as_ref() {
+ Some(allocator) if allocator.is_empty() => !allocator.matches_layout(layout)?,
+ _ => false,
+ };
+ if replace_empty_pool {
+ self.vram_slots = None;
+ }
+
+ if let Some(allocator) = self.vram_slots.as_mut() {
+ return allocator.alloc(layout);
+ }
+
+ let mut allocator = VgpuVramSlotAllocator::new(mm, layout)?;
+ let slot = allocator.alloc(layout)?;
+ self.vram_slots = Some(allocator);
+ Ok(slot)
+ }
+
+ fn release_vram_slot(&mut self, slot: VgpuVramSlot) {
+ let Some(allocator) = self.vram_slots.as_mut() else {
+ // A live slot proves that its pool exists. If that invariant is ever broken,
+ // leaking the slot is safer than allowing its backing VRAM to be reused.
+ core::mem::forget(slot);
+ return;
+ };
+ allocator.release(slot);
+ }
+
+ /// Allocate resources and register a new inactive vGPU instance.
+ pub(crate) fn allocate_instance(
+ &mut self,
+ mm: &GpuMm<'_>,
+ vgpu: &VgpuManager<'gpu>,
+ info: InstanceInfo,
+ ) -> Result<Gfid> {
+ let InstanceInfo {
+ gfid,
+ dbdf,
+ vgpu_type,
+ vm_pid,
+ } = info;
+
+ if self
+ .instances
+ .iter()
+ .any(|instance| instance.gfid == gfid || instance.dbdf == dbdf)
+ {
+ return Err(EEXIST);
+ }
+ let profile_instances = self
+ .instances
+ .iter()
+ .filter(|instance| instance.vgpu_type.vgpu_type_id == vgpu_type.vgpu_type_id)
+ .count();
+ if vgpu_type.max_instance == 0
+ || profile_instances
+ >= usize::try_from(vgpu_type.max_instance).map_err(|_| EOVERFLOW)?
+ {
+ return Err(ENOSPC);
+ }
+ // Reserve registry capacity before acquiring resources so publishing
+ // the completed instance cannot fail due to memory pressure.
+ self.instances.reserve(1, GFP_KERNEL)?;
+
+ let num_chid = vgpu
+ .total_channels()
+ .ok_or(ENODEV)?
+ .checked_div(vgpu_type.max_instance)
+ .filter(|count| *count != 0)
+ .ok_or(EINVAL)?;
+ let chids = vgpu.chid_pool.reserve_ids(
+ NonZeroUsize::new(usize::try_from(num_chid).map_err(|_| EOVERFLOW)?).ok_or(EINVAL)?,
+ Alignment::SZ_1,
+ )?;
+ let layout = VgpuVramLayout {
+ type_id: vgpu_type.vgpu_type_id,
+ max_slots: vgpu_type.max_instance,
+ fb_size: vgpu_type.fb_length,
+ heap_size: vgpu_type.gsp_heap_size,
+ fb_align: vgpu.vmmu_segment_size().ok_or(ENODEV)?,
+ };
+ let vram_slot = self.alloc_vram_slot(mm, layout)?;
+
+ let instance = VgpuInstance {
+ gfid,
+ dbdf,
+ vgpu_type,
+ vm_pid,
+ chids,
+ num_plugin_channels: 3,
+ vram_slot,
+ };
+ match self.instances.push_within_capacity(instance) {
+ Ok(()) => Ok(gfid),
+ Err(error) => {
+ let VgpuInstance { vram_slot, .. } = error.0;
+ self.release_vram_slot(vram_slot);
+ Err(EIO)
+ }
+ }
+ }
+
+ /// Remove an instance and release its channel and VRAM reservations.
+ pub(crate) fn destroy_instance(&mut self, gfid: Gfid) -> Result {
+ let index = self
+ .instances
+ .iter()
+ .position(|instance| instance.gfid == gfid)
+ .ok_or(ENOENT)?;
+ let instance = self.instances.remove(index).map_err(|_| EIO)?;
+ let VgpuInstance { vram_slot, .. } = instance;
+ self.release_vram_slot(vram_slot);
+ Ok(())
+ }
+}
+
+/// Query the vGPU type assigned to a VF by its DBDF.
+#[expect(dead_code)]
+pub(crate) fn query_assigned_vf_type(cmdq: &Cmdq, bar: Bar0<'_>, dbdf: Dbdf) -> Result<u32> {
+ let request = u64::from(dbdf.into_raw()).to_le_bytes();
+ let response =
+ cmdq.send_gmc_and_receive(bar, gmc::VGPU_MGMT_QUERY_ASSIGNED_VF, &request, 64)?;
+ if response.status != 0 {
+ return Err(EIO);
+ }
+ let bytes = response.payload.get(..4).ok_or(ENODEV)?;
+ Ok(u32::from_le_bytes(bytes.try_into().map_err(|_| EINVAL)?))
+}
+
+/// Query and decode one vGPU type using the typed NVKV schema.
+#[expect(dead_code)]
+pub(crate) fn query_vgpu_type(cmdq: &Cmdq, bar: Bar0<'_>, type_id: u32) -> Result<VgpuType> {
+ let response = cmdq.send_gmc_and_receive(
+ bar,
+ gmc::VGPU_MGMT_QUERY_PROPERTIES,
+ &type_id.to_le_bytes(),
+ 4096,
+ )?;
+ if response.status != 0 {
+ return Err(EIO);
+ }
+
+ let properties = decode_vgpu_properties(&response.payload)?;
+ if properties.type_id != type_id || properties.max_instance == 0 {
+ return Err(EINVAL);
+ }
+ Ok(VgpuType::from_properties(&properties))
+}
diff --git a/drivers/gpu/nova-core/vgpu/mod.rs b/drivers/gpu/nova-core/vgpu/mod.rs
index a96d0018fa3d..320230ddd1dd 100644
--- a/drivers/gpu/nova-core/vgpu/mod.rs
+++ b/drivers/gpu/nova-core/vgpu/mod.rs
@@ -3,10 +3,17 @@
use core::num::NonZero;
+pub(crate) mod consts;
+pub(crate) mod instance;
+
+pub(crate) use self::instance::VgpuInstances;
+
use kernel::{
device,
+ new_mutex,
pci,
- prelude::*, //
+ prelude::*,
+ sync::Mutex, //
};
use crate::{
@@ -38,9 +45,12 @@ pub(crate) enum VgpuState {
}
/// vGPU state manager.
+#[pin_data]
pub(crate) struct VgpuManager<'gpu> {
+ /// Live vGPU instances owned by this manager.
+ #[pin]
+ instances: Mutex<VgpuInstances<'gpu>>,
/// Channel ID pool the per-VF areas are reserved from.
- #[expect(dead_code)]
pub(crate) chid_pool: &'gpu ChannelIdPool,
state: VgpuState,
vmmu_segment_size: Option<u64>,
@@ -50,19 +60,20 @@ pub(crate) struct VgpuManager<'gpu> {
impl<'gpu> VgpuManager<'gpu> {
/// Creates an empty vGPU manager for initialization during GPU construction.
- pub(crate) const fn new(chid_pool: &'gpu ChannelIdPool) -> Self {
- Self {
+ pub(crate) fn new(chid_pool: &'gpu ChannelIdPool) -> impl PinInit<Self> + 'gpu {
+ pin_init!(Self {
+ instances <- new_mutex!(VgpuInstances::new(), "nova-core::vgpu-instances"),
chid_pool,
state: VgpuState::Disabled,
vmmu_segment_size: None,
total_channels: None,
fifo_engine_list: None,
- }
+ })
}
/// Detects and stores vGPU state before GSP boot.
pub(crate) fn detect_state(
- &mut self,
+ self: Pin<&mut Self>,
pdev: &pci::Device<device::Core<'_>>,
chipset: Chipset,
fsp: Option<&mut Fsp<'_>>,
@@ -91,7 +102,7 @@ pub(crate) fn detect_state(
}
})();
- self.state = state.unwrap_or_else(|e| {
+ let state = state.unwrap_or_else(|e| {
dev_warn!(
pdev,
"vGPU state detection failed: {:?}; disabling vGPU\n",
@@ -99,7 +110,9 @@ pub(crate) fn detect_state(
);
VgpuState::Disabled
});
- dev_dbg!(pdev, "vGPU state: {:?}\n", self.state);
+ let this = self.project();
+ *this.state = state;
+ dev_dbg!(pdev, "vGPU state: {:?}\n", state);
}
/// Returns the detected vGPU state for this boot.
@@ -109,26 +122,31 @@ pub(crate) fn state(&self) -> VgpuState {
/// Initializes the runtime parameters returned by GSP_INIT.
pub(crate) fn init(
- &mut self,
+ self: Pin<&mut Self>,
fifo_engine_list: &FifoEngineList,
vmmu_segment_size: u64,
total_channels: u32,
) {
- if matches!(self.state, VgpuState::Enabled { .. }) {
- self.vmmu_segment_size = Some(vmmu_segment_size);
- self.total_channels = Some(total_channels);
- self.fifo_engine_list = Some(*fifo_engine_list);
+ let this = self.project();
+ if matches!(*this.state, VgpuState::Enabled { .. }) {
+ *this.vmmu_segment_size = Some(vmmu_segment_size);
+ *this.total_channels = Some(total_channels);
+ *this.fifo_engine_list = Some(*fifo_engine_list);
}
}
- /// Returns the firmware-reported VMMU segment size when vGPU is enabled.
+ /// Returns the live-instance registry.
#[expect(dead_code)]
+ pub(crate) fn instances(&self) -> &Mutex<VgpuInstances<'gpu>> {
+ &self.instances
+ }
+
+ /// Returns the firmware-reported VMMU segment size when vGPU is enabled.
pub(crate) const fn vmmu_segment_size(&self) -> Option<u64> {
self.vmmu_segment_size
}
/// Returns the number of channel IDs available to vGPU instances.
- #[expect(dead_code)]
pub(crate) const fn total_channels(&self) -> Option<u32> {
self.total_channels
}
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 06/13] gpu: nova-core: gsp: add GMC transaction helpers
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
` (4 preceding siblings ...)
2026-09-05 8:11 ` [PATCH 05/13] gpu: nova-core: vgpu: add instance create/destroy Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 07/13] gpu: nova-core: vgpu: add vGPU bootload Zhi Wang
` (6 subsequent siblings)
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
Match GMC replies to requests by both the masked command identifier and
sequence number instead of accepting the next GMC message as the reply.
Use one deadline while stale responses and interleaved messages are
consumed.
Pass the GMC sequence through event dispatch, and add helpers for
status-only replies and commands completed by asynchronous events.
A GMC transaction holds the shared command queue lock while waiting.
Consume and dispatch a valid interleaved RM RPC frame instead of treating
it as malformed GMC framing, then continue waiting for the GMC message.
The GSP-to-CPU queue carries both RM RPC and GMC messages. The threaded
drain predates GMC support and parses every pending entry as RM RPC.
Consequently, an asynchronous GMC message fails RPC framing validation,
poisons the queue, and leaves the CPU read pointer stuck on that entry.
Use the common classifier for both wire formats. It consumes and
dispatches valid RM RPC messages while returning GMC messages to the
caller. Advance the read pointer for returned GMC messages so the drain
can continue.
The drain holds the command queue lock, so no synchronous GMC transaction
can be waiting for a returned message at the same time. Such a message is
therefore unsolicited or stale and can be consumed without dispatch.
Co-developed-by: Alok Kumar <alkumar@nvidia.com>
Signed-off-by: Alok Kumar <alkumar@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/gsp/cmdq.rs | 311 ++++++++++++++++++--------
drivers/gpu/nova-core/gsp/commands.rs | 30 ++-
drivers/gpu/nova-core/gsp/fw.rs | 22 ++
3 files changed, 265 insertions(+), 98 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index e472ec94691d..8f204f71a36a 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -52,6 +52,7 @@
driver::Bar0,
gsp::{
fw::{
+ GmcApiHeader,
GmcCommand,
GspGmcMsgElement,
GspMsgElement,
@@ -648,8 +649,8 @@ fn receive_msg<M: MessageFromGsp>(&self, bar: Bar0<'_>, timeout: Delta) -> Resul
self.inner.lock().receive_msg(bar, timeout, None)
}
- /// Receives one GMC element from the GSP and passes its command id, the `max_resp_or_status`
- /// field, and the raw payload slices to `handler`.
+ /// Receives one GMC element from the GSP and passes its header and raw payload slices to
+ /// `handler`.
///
/// This method may sleep while waiting. The [`CmdqInner`] mutex stays locked across the wait
/// and across the `handler` call, so `handler` must not call back into this [`Cmdq`].
@@ -659,7 +660,7 @@ pub(crate) fn receive_gmc_and_dispatch<R>(
&self,
bar: Bar0<'_>,
timeout: Delta,
- handler: impl FnOnce(u32, u32, &[u8], &[u8]) -> (Option<R>, QueuePointers),
+ handler: impl FnOnce(&GmcApiHeader, &[u8], &[u8]) -> (Option<R>, QueuePointers),
) -> Result<Option<R>> {
self.inner
.lock()
@@ -686,9 +687,10 @@ pub(crate) fn send_gmc_no_wait(
self.inner
.lock()
.send_gmc(bar, command_id, payload, max_response_size)
+ .map(|_| ())
}
- /// Sends a GMC API command and waits for its response.
+ /// Sends a GMC API command and waits for its matching response.
///
/// The queue stays locked for the complete transaction. A single deadline bounds all queue
/// elements observed while waiting.
@@ -698,26 +700,59 @@ pub(crate) fn send_gmc_and_receive(
command_id: u32,
payload: &[u8],
max_response_size: u32,
+ ) -> Result<GmcResponse> {
+ self.send_gmc_and_receive_timeout(
+ bar,
+ command_id,
+ payload,
+ max_response_size,
+ Self::RECEIVE_TIMEOUT,
+ )
+ }
+
+ /// Sends a GMC API command and waits up to `timeout` for its matching response.
+ pub(crate) fn send_gmc_and_receive_timeout(
+ &self,
+ bar: Bar0<'_>,
+ command_id: u32,
+ payload: &[u8],
+ max_response_size: u32,
+ timeout: Delta,
) -> Result<GmcResponse> {
let mut inner = self.inner.lock();
- inner.send_gmc(bar, command_id, payload, max_response_size)?;
+ let expected_sequence = inner.send_gmc(bar, command_id, payload, max_response_size)?;
+ let dev = inner.dev.clone();
- let deadline = Instant::<Monotonic>::now() + Self::RECEIVE_TIMEOUT;
+ let deadline = Instant::<Monotonic>::now() + timeout;
loop {
let remaining = deadline - Instant::<Monotonic>::now();
if remaining.is_negative() {
return Err(ETIMEDOUT);
}
- let response = inner.receive_gmc_and_dispatch(
+ let response = match inner.receive_gmc_and_dispatch(
bar,
remaining,
- |received_command, status, payload_0, payload_1| {
- if received_command != command_id {
+ |header, payload_0, payload_1| {
+ if !header.is_response_to(command_id, expected_sequence) {
+ let kind = if header.is_response() {
+ "response"
+ } else {
+ "event"
+ };
+ dev_dbg!(
+ &dev,
+ "GSP GMC: skip {} seq {} cmd {:#x}; want response seq {} cmd {:#x}\n",
+ kind,
+ header.sequence_number(),
+ header.command_id(),
+ expected_sequence,
+ command_id,
+ );
return (None, QueuePointers::Unchanged);
}
- let response = (|| {
+ let response: Result<GmcResponse> = (|| {
let mut payload = KVec::with_capacity(
payload_0
.len()
@@ -728,14 +763,18 @@ pub(crate) fn send_gmc_and_receive(
payload.extend_from_slice(payload_0, GFP_KERNEL)?;
payload.extend_from_slice(payload_1, GFP_KERNEL)?;
Ok(GmcResponse {
- status,
+ status: header.max_resp_or_status,
payload,
})
})();
(Some(response), QueuePointers::Unchanged)
},
- )?;
+ ) {
+ Ok(response) => response,
+ Err(ERANGE) => continue,
+ Err(error) => return Err(error),
+ };
if let Some(response) = response {
return response;
@@ -743,6 +782,81 @@ pub(crate) fn send_gmc_and_receive(
}
}
+ /// Sends a synchronous GMC command and checks its status-only reply.
+ pub(crate) fn send_gmc_and_check_status(
+ &self,
+ bar: Bar0<'_>,
+ command_id: u32,
+ payload: &[u8],
+ ) -> Result {
+ let response = self.send_gmc_and_receive(bar, command_id, payload, 0)?;
+ if response.status == 0 {
+ Ok(())
+ } else {
+ Err(EIO)
+ }
+ }
+
+ /// Sends an asynchronous GMC command and waits atomically for its event.
+ ///
+ /// The command queue remains locked from the send through the matching
+ /// event, preventing another transaction from consuming its completion.
+ /// Interleaved GMC events are passed to `handler`; responses are consumed
+ /// while waiting. The timeout is shared by the complete operation.
+ pub(crate) fn send_gmc_and_wait_event(
+ &self,
+ bar: Bar0<'_>,
+ command_id: u32,
+ payload: &[u8],
+ timeout: Delta,
+ mut predicate: impl FnMut(u32, u32, u64, &[u8], &[u8]) -> Result<bool>,
+ mut handler: impl FnMut(u32, u32, u64, &[u8], &[u8]) -> Result,
+ ) -> Result {
+ let mut inner = self.inner.lock();
+ inner.send_gmc(bar, command_id, payload, 0)?;
+ let deadline = Instant::<Monotonic>::now() + timeout;
+
+ loop {
+ let remaining = deadline - Instant::<Monotonic>::now();
+ if remaining.is_negative() {
+ return Err(ETIMEDOUT);
+ }
+
+ let matched = match inner.receive_gmc_and_dispatch(
+ bar,
+ remaining,
+ |header, payload_0, payload_1| {
+ let result: Result<bool> = (|| {
+ if header.is_response() {
+ return Ok(false);
+ }
+
+ let command = header.command_id();
+ let status = header.max_resp_or_status;
+ let sequence = header.sequence_number();
+ if predicate(command, status, sequence, payload_0, payload_1)? {
+ return Ok(true);
+ }
+
+ handler(command, status, sequence, payload_0, payload_1)?;
+ Ok(false)
+ })();
+ (Some(result), QueuePointers::Unchanged)
+ },
+ ) {
+ Ok(matched) => matched,
+ Err(ERANGE) => continue,
+ Err(error) => return Err(error),
+ };
+
+ if let Some(matched) = matched {
+ if matched? {
+ return Ok(());
+ }
+ }
+ }
+ }
+
/// Waits for an unsolicited GSP event of type `M`, dispatching any other event that arrives
/// first.
///
@@ -772,8 +886,8 @@ pub(crate) fn await_msg<M: MessageFromGsp>(&self, bar: Bar0<'_>) -> Result<M>
/// Drains and dispatches every message currently pending in the GSP-to-CPU queue.
///
- /// Routes each message the GSP has already posted through [`CmdqInner::dispatch_event`] and
- /// returns without waiting for more.
+ /// Dispatches pending RM RPC messages as events, consumes pending GMC messages, and returns
+ /// without waiting for more.
///
/// # Errors
///
@@ -936,9 +1050,10 @@ fn send_gmc(
command_id: u32,
payload: &[u8],
max_response_size: u32,
- ) -> Result {
+ ) -> Result<u64> {
let rpc_seq = self.rpc_seq;
self.rpc_seq = self.rpc_seq.wrapping_add(1);
+ let sequence = u64::from(rpc_seq);
let dst = self.gsp_mem.allocate_command::<GspGmcMsgElement>(
bar,
@@ -946,12 +1061,8 @@ fn send_gmc(
Self::ALLOCATE_TIMEOUT,
)?;
- let msg_element = GspGmcMsgElement::init(
- command_id,
- u64::from(rpc_seq),
- payload.len(),
- max_response_size,
- );
+ let msg_element =
+ GspGmcMsgElement::init(command_id, sequence, payload.len(), max_response_size);
// SAFETY: `dst.header` points to a valid, writable `GspGmcMsgElement` region.
unsafe {
msg_element.__init(core::ptr::from_mut(dst.header))?;
@@ -963,7 +1074,7 @@ fn send_gmc(
dev_dbg!(
&self.dev,
"GSP GMC: send: seq# {}, command={}, length=0x{:x}\n",
- rpc_seq,
+ sequence,
GmcCommand(command_id),
dst.header.length(),
);
@@ -971,7 +1082,7 @@ fn send_gmc(
let elem_count = dst.header.element_count();
DmaGspMem::advance_cpu_write_ptr_v2(bar, elem_count);
- Ok(())
+ Ok(sequence)
}
/// Wait for a message to become available on the message queue.
@@ -1022,6 +1133,12 @@ fn wait_for_msg(&self, bar: Bar0<'_>, timeout: Delta) -> Result<GspMessage<'_>>
return Err(e);
}
+ if !header.is_rm_rpc() {
+ dev_err!(&self.dev, "GSP RPC: receive: invalid NVDM type\n");
+ self.poisoned.set(true);
+ return Err(EIO);
+ }
+
let payload_length = header.payload_length();
// Check that the driver read area is large enough for the message.
@@ -1084,6 +1201,26 @@ fn advance_rx_event_seq(&mut self, function: Result<MsgFunction, u32>) {
}
}
+ /// Consumes and dispatches the RM RPC element currently at the queue head.
+ fn consume_and_dispatch_rpc(&mut self, bar: Bar0<'_>) -> Result {
+ let (function, seq, length) = {
+ let message = self.wait_for_msg(bar, Delta::ZERO)?;
+
+ (
+ message.header.function(),
+ message.header.sequence(),
+ message.header.length(),
+ )
+ };
+
+ self.log_received(function, seq, length);
+ let pages = u32::try_from(length.div_ceil(GSP_PAGE_SIZE))?;
+ DmaGspMem::advance_cpu_read_ptr_v2(bar, pages);
+ self.advance_rx_event_seq(function);
+ self.dispatch_event(function, seq);
+ Ok(())
+ }
+
/// Receive a message from the GSP.
///
/// The expected message type is given by the `M` generic parameter. With `expected_seq` set,
@@ -1203,32 +1340,22 @@ fn dispatch_event(&self, function: Result<MsgFunction, u32>, seq: u32) {
/// Drains and dispatches all messages currently pending in the GSP-to-CPU queue.
///
- /// Processes whatever the GSP has already posted, dispatching each message as an event, and
- /// stops once the queue is empty. There is no awaited reply during a drain, so every message
- /// is routed to [`Self::dispatch_event`].
+ /// RM RPC messages are dispatched as events. GMC messages are unsolicited while the drain owns
+ /// the command queue lock, so they are consumed without dispatch.
///
/// # Errors
///
/// Returns the receive error that stopped the drain, in particular the `EIO` of a queue
- /// poisoned by corrupt framing (see [`Self::wait_for_msg`]).
+ /// poisoned by framing that is invalid for both GMC and RM RPC (see
+ /// [`Self::receive_gmc_and_dispatch`]).
fn drain(&mut self, bar: Bar0<'_>) -> Result {
while !self.gsp_mem.driver_read_area_v2(bar).0.is_empty() {
- // A message is available, so this returns without waiting.
- let msg = self.wait_for_msg(bar, Delta::ZERO)?;
- let function = msg.header.function();
- let seq = msg.header.sequence();
- let length = msg.header.length();
-
- self.log_received(function, seq, length);
-
- let pages = u32::try_from(length.div_ceil(GSP_PAGE_SIZE)).map_err(|_| {
- dev_err!(&self.dev, "GSP drain: message length overflow\n");
- EIO
- })?;
-
- DmaGspMem::advance_cpu_read_ptr_v2(bar, pages);
- self.advance_rx_event_seq(function);
- self.dispatch_event(function, seq);
+ match self.receive_gmc_and_dispatch::<()>(bar, Delta::ZERO, |_, _, _| {
+ (None, QueuePointers::Unchanged)
+ }) {
+ Ok(_) | Err(ERANGE) => {}
+ Err(error) => return Err(error),
+ }
}
Ok(())
@@ -1269,7 +1396,14 @@ fn wait_for_gmc_msg(&self, bar: Bar0<'_>, timeout: Delta) -> Result<GmcMessage<'
};
// Checked before any length field is read, since bad framing leaves them untrusted.
- if let Err(e) = header.validate_framing() {
+ let framing = header.validate_common_framing().and_then(|()| {
+ if header.is_gmc_api() {
+ header.validate_framing()
+ } else {
+ Ok(())
+ }
+ });
+ if let Err(e) = framing {
dev_err!(
&self.dev,
"GSP GMC: receive: bad MCTP framing, declared length {}\n",
@@ -1306,12 +1440,11 @@ fn wait_for_gmc_msg(&self, bar: Bar0<'_>, timeout: Delta) -> Result<GmcMessage<'
})
}
- /// Receive the next GMC event from the GSP and dispatch it through a handler.
+ /// Receive the next GMC element from the GSP and dispatch it through a handler.
///
- /// The handler receives the GMC command id, the header's `max_resp_or_status` field, and the
- /// raw payload slices that follow the [`super::fw::GmcApiHeader`] (two slices because the
- /// circular buffer may wrap). It returns `None` for an element it does not handle, paired
- /// with the [`QueuePointers`] state it left behind.
+ /// The handler receives the GMC header and the raw payload slices that follow it (two slices
+ /// because the circular buffer may wrap). It returns `None` for an element it does not handle,
+ /// paired with the [`QueuePointers`] state it left behind.
///
/// `max_resp_or_status` is a union: GSP-RM writes an `NV_STATUS` there when the element is a
/// response, and the maximum response size when it is a request. Only a handler that knows
@@ -1320,23 +1453,29 @@ fn wait_for_gmc_msg(&self, bar: Bar0<'_>, timeout: Delta) -> Result<GmcMessage<'
/// Where [`Self::receive_msg`] keys on [`MsgFunction`], this keys on the GMC command id,
/// which is the form the r000 firmware uses for boot events.
///
- /// Returns `Ok(None)` when nothing claimed the element, either because it is not a GMC
- /// element or because the handler declined it. The read pointer is advanced past the element
- /// on every path except a handler reporting [`QueuePointers::Reset`], which has already
- /// returned both pointers to zero.
+ /// Returns `Ok(None)` when the handler declines a GMC element. A valid interleaved RM RPC
+ /// element is consumed and dispatched before this method returns `ERANGE`. The read pointer is
+ /// advanced past the element on every path except a handler reporting [`QueuePointers::Reset`],
+ /// which has already returned both pointers to zero.
///
/// # Errors
///
/// - `ETIMEDOUT` if `timeout` has elapsed before any message becomes available.
/// - `EIO` if the queue is poisoned or the element fails framing validation (see
/// [`Self::wait_for_gmc_msg`]).
+ /// - `ERANGE` if a valid interleaved RM RPC element was consumed and dispatched.
fn receive_gmc_and_dispatch<R>(
&mut self,
bar: Bar0<'_>,
timeout: Delta,
- handler: impl FnOnce(u32, u32, &[u8], &[u8]) -> (Option<R>, QueuePointers),
+ handler: impl FnOnce(&GmcApiHeader, &[u8], &[u8]) -> (Option<R>, QueuePointers),
) -> Result<Option<R>> {
let message = self.wait_for_gmc_msg(bar, timeout)?;
+ if !message.header.is_gmc_api() {
+ self.consume_and_dispatch_rpc(bar)?;
+ return Err(ERANGE);
+ }
+
let header = message.header;
let length = header.length();
@@ -1345,45 +1484,37 @@ fn receive_gmc_and_dispatch<R>(
let cpu_write_ptr = DmaGspMem::cpu_write_ptr_v2(bar);
let cpu_read_ptr = DmaGspMem::cpu_read_ptr_v2(bar);
- // The RPC and GMC elements share every field through `nvdm_header`, so `gmc` holds an
- // RPC header rather than a GMC one unless the NVDM type says otherwise.
- let (result, queue_pointers) = if !header.is_gmc_api() {
- dev_warn!(&self.dev, "GSP GMC: dropping non-GMC queue element\n");
- (None, QueuePointers::Unchanged)
- } else if num::u32_as_usize(header.gmc.size) != header.payload_length() {
- // GSP-RM sends the element as the GMC header plus `size` bytes, so the two lengths
- // describe the same payload and a handler cannot tell which one to believe.
- dev_err!(
- &self.dev,
- "GSP GMC: payload is {} bytes, transport declares {}\n",
- header.gmc.size,
- header.payload_length(),
- );
- (None, QueuePointers::Unchanged)
- } else {
- let command_id = header.gmc.command_id();
- let kind = if header.gmc.is_response() {
- "response"
+ let (result, queue_pointers) =
+ if num::u32_as_usize(header.gmc.size) != header.payload_length() {
+ // GSP-RM sends the element as the GMC header plus `size` bytes, so the two lengths
+ // describe the same payload and a handler cannot tell which one to believe.
+ dev_err!(
+ &self.dev,
+ "GSP GMC: payload is {} bytes, transport declares {}\n",
+ header.gmc.size,
+ header.payload_length(),
+ );
+ (None, QueuePointers::Unchanged)
} else {
- "event"
- };
+ let command_id = header.gmc.command_id();
+ let sequence = header.gmc.sequence_number();
+ let kind = if header.gmc.is_response() {
+ "response"
+ } else {
+ "event"
+ };
- dev_dbg!(
- &self.dev,
- "GSP GMC: {}: seq# {}, command={}, length=0x{:x}\n",
- kind,
- header.gmc.sequence_number(),
- GmcCommand(command_id),
- length,
- );
+ dev_dbg!(
+ &self.dev,
+ "GSP GMC: {}: seq# {}, command={}, length=0x{:x}\n",
+ kind,
+ sequence,
+ GmcCommand(command_id),
+ length,
+ );
- handler(
- command_id,
- header.gmc.max_resp_or_status,
- message.contents.0,
- message.contents.1,
- )
- };
+ handler(&header.gmc, message.contents.0, message.contents.1)
+ };
let pages = u32::try_from(length.div_ceil(GSP_PAGE_SIZE))?;
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 417df31988c2..bdb358a19738 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -13,6 +13,10 @@
device,
pci,
prelude::*,
+ time::{
+ Instant,
+ Monotonic, //
+ },
transmute::AsBytes, //
};
@@ -219,20 +223,27 @@ pub(crate) fn gsp_init(
GSP_INIT_MAX_RESPONSE_SIZE,
)?;
+ let deadline = Instant::<Monotonic>::now() + Cmdq::RECEIVE_TIMEOUT;
loop {
- let reply = cmdq.receive_gmc_and_dispatch(
- bar,
- Cmdq::RECEIVE_TIMEOUT,
- |command_id, max_resp_or_status, payload_0, payload_1| {
- if command_id == GMCAPI_CMD_GSP_INIT {
+ let remaining = deadline - Instant::<Monotonic>::now();
+ if remaining.is_negative() {
+ return Err(ETIMEDOUT);
+ }
+
+ let reply =
+ match cmdq.receive_gmc_and_dispatch(bar, remaining, |header, payload_0, payload_1| {
+ let command_id = header.command_id();
+ if header.is_response() && command_id == GMCAPI_CMD_GSP_INIT {
(
Some(decode_gsp_init_reply(
- max_resp_or_status,
+ header.max_resp_or_status,
payload_0,
payload_1,
)),
QueuePointers::Unchanged,
)
+ } else if header.is_response() {
+ (None, QueuePointers::Unchanged)
} else {
// A boot event. Keep waiting for the reply unless handling it failed.
match on_boot_event(command_id, payload_0) {
@@ -242,8 +253,11 @@ pub(crate) fn gsp_init(
Err(e) => (Some(Err(e)), QueuePointers::Reset),
}
}
- },
- )?;
+ }) {
+ Ok(reply) => reply,
+ Err(ERANGE) => continue,
+ Err(error) => return Err(error),
+ };
if let Some(reply) = reply {
return reply;
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index f8f7f85d2df0..f8c6fc5ea8a7 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -596,6 +596,11 @@ pub(crate) fn validate_framing(&self) -> Result {
)
}
+ /// Returns `true` if the NVDM header routes this element to the RM RPC dispatcher.
+ pub(crate) fn is_rm_rpc(&self) -> bool {
+ self.nvdm_header.validate(NvdmType::RmRpc)
+ }
+
// Returns the sequence number of the message.
pub(crate) fn sequence(&self) -> u32 {
self.rpc.sequence
@@ -735,6 +740,13 @@ pub(crate) fn is_response(&self) -> bool {
self.command & GMCAPI_COMMAND_FLAGS_RESPONSE != 0
}
+ /// Returns `true` if this header is the response to `command_id` and `sequence`.
+ pub(crate) fn is_response_to(&self, command_id: u32, sequence: u64) -> bool {
+ self.is_response()
+ && self.command_id() == (command_id & GMCAPI_COMMAND_ID_MASK)
+ && self.sequence == sequence
+ }
+
/// Returns [`Self::sequence`] with the GSP-initiated-event bit cleared.
pub(crate) fn sequence_number(&self) -> u64 {
self.sequence & !GMC_EVENT_SEQUENCE_BASE
@@ -911,6 +923,16 @@ pub(crate) fn validate_framing(&self) -> Result {
)
}
+ pub(crate) fn validate_common_framing(&self) -> Result {
+ validate_mctp_framing(
+ self.mctp_magic,
+ self.mctp_payload_size,
+ self.mctp_header,
+ self.nvdm_header,
+ core::mem::offset_of!(Self, gmc),
+ )
+ }
+
/// Returns `true` if the NVDM header routes this element to the GSP's GMC dispatch.
///
/// A [`GspMsgElement`] and a [`GspGmcMsgElement`] share every field through `nvdm_header`,
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 07/13] gpu: nova-core: vgpu: add vGPU bootload
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
` (5 preceding siblings ...)
2026-09-05 8:11 ` [PATCH 06/13] gpu: nova-core: gsp: add GMC transaction helpers Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 08/13] gpu: nova-core: vgpu: implement PluginRpc channel and config params Zhi Wang
` (5 subsequent siblings)
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
Implement the GMCAPI VGPU_BOOTLOAD command that boots the GSP plugin
for a vGPU instance, together with the VGPU_SHUTDOWN and VGPU_CLEANUP
teardown sequence.
Encode the typed channel map, framebuffer, management heap and log
locations, then poll the PluginRpc BAR1 marker for boot completion.
Co-developed-by: Alok Kumar <alkumar@nvidia.com>
Signed-off-by: Alok Kumar <alkumar@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/gsp/commands.rs | 2 +
drivers/gpu/nova-core/gsp/fw.rs | 18 ++-
drivers/gpu/nova-core/gsp/fw/commands.rs | 60 +++++++
drivers/gpu/nova-core/mm/vram.rs | 3 -
drivers/gpu/nova-core/vgpu/bootload.rs | 162 +++++++++++++++++++
drivers/gpu/nova-core/vgpu/consts.rs | 8 +
drivers/gpu/nova-core/vgpu/fw.rs | 193 +++++++++++++++++++++++
drivers/gpu/nova-core/vgpu/instance.rs | 100 ++++++++++--
drivers/gpu/nova-core/vgpu/mod.rs | 2 +
drivers/gpu/nova-core/vgpu/plugin_rpc.rs | 65 ++++++++
drivers/gpu/nova-core/vgpu/vram.rs | 2 -
11 files changed, 598 insertions(+), 17 deletions(-)
create mode 100644 drivers/gpu/nova-core/vgpu/bootload.rs
create mode 100644 drivers/gpu/nova-core/vgpu/plugin_rpc.rs
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index bdb358a19738..481ec8e221ce 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -54,6 +54,8 @@
};
pub(crate) use fw::commands::{
+ encode_vgpu_bootload,
+ ChannelMapEntry,
Dbdf,
VgpuProperties, //
};
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index f8c6fc5ea8a7..d68b790533af 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -7,8 +7,24 @@
/// Raw firmware declarations used by vGPU management.
pub(crate) mod vgpu_bindings {
pub(crate) use super::r000_00::{
+ GMCAPI_COMMANDS_GMCAPI_CMD_BOOTLOAD_GSP_VGPU_PLUGIN_TASK,
+ GMCAPI_COMMANDS_GMCAPI_CMD_CLEANUP_GSP_VGPU_PLUGIN_RESOURCES,
GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_ASSIGNED_VF_VGPU_TYPE,
- GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_VGPU_PROPERTIES, //
+ GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_VGPU_PROPERTIES,
+ GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK,
+ GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK_COMPLETE,
+ GSP_PLUGIN_BOOTLOADED,
+ VGPU_CPU_GSP_COMMUNICATION_BUFF_TOTAL_SIZE,
+ VGPU_CPU_GSP_CTRL_BUFF_REGION,
+ VGPU_CPU_GSP_CTRL_BUFF_REGION_SIZE,
+ VGPU_CPU_GSP_ERROR_BUFF_REGION_SIZE,
+ VGPU_CPU_GSP_GUEST_RPC_TRACE_BUFF_REGION_SIZE,
+ VGPU_CPU_GSP_INIT_TASK_LOG_BUFF_REGION_SIZE,
+ VGPU_CPU_GSP_KERNEL_TASK_LOG_BUFF_REGION_SIZE,
+ VGPU_CPU_GSP_MESSAGE_BUFF_REGION_SIZE,
+ VGPU_CPU_GSP_MIGRATION_BUFF_REGION_SIZE,
+ VGPU_CPU_GSP_RESPONSE_BUFF_REGION_SIZE,
+ VGPU_CPU_GSP_VGPU_TASK_LOG_BUFF_REGION_SIZE, //
};
}
diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs
index 0603fbde172f..fe4a7af88ec7 100644
--- a/drivers/gpu/nova-core/gsp/fw/commands.rs
+++ b/drivers/gpu/nova-core/gsp/fw/commands.rs
@@ -537,6 +537,13 @@ pub(crate) struct ChannelMapEntry(u64) {
impl ChannelMapEntry {
const KEY: KeyId = 0x1001;
+
+ pub(crate) fn new(engine_type: usize, index: u32, chid_offset: u32) -> Result<Self> {
+ Self::zeroed()
+ .try_with_engine_type(u64::try_from(engine_type).map_err(|_| EOVERFLOW)?)
+ .and_then(|entry| entry.try_with_index(u64::from(index)))
+ .and_then(|entry| entry.try_with_chid_offset(u64::from(chid_offset)))
+ }
}
impl Encodeable for KVVec<ChannelMapEntry> {
@@ -611,6 +618,59 @@ impl VgpuBootloadRequest {
const MIG_RM_HEAP_LENGTH_KEY: KeyId = 0x100E;
}
+/// Encodes a `VGPU_BOOTLOAD` request using the typed NVKV schema.
+#[expect(clippy::too_many_arguments)]
+pub(crate) fn encode_vgpu_bootload(
+ dbdf: Dbdf,
+ gfid: u32,
+ vgpu_type: u32,
+ vm_pid: u32,
+ num_channels: u32,
+ num_plugin_channels: u32,
+ channel_mapping: KVVec<ChannelMapEntry>,
+ guest_fb_address: u64,
+ guest_fb_length: u64,
+ plugin_heap_address: u64,
+ plugin_heap_length: u64,
+ ctrl_buffer_offset: u64,
+ init_log_address: u64,
+ init_log_size: u64,
+ vgpu_log_address: u64,
+ vgpu_log_size: u64,
+ kernel_log_address: u64,
+ kernel_log_size: u64,
+) -> Result<KVVec<u64>> {
+ let request = VgpuBootloadRequest {
+ dbdf: dbdf.into(),
+ gfid: gfid.into(),
+ vgpu_type: vgpu_type.into(),
+ vm_pid: vm_pid.into(),
+ swizz_id: SwizzId::WHOLE_GPU.into(),
+ num_channels: num_channels.into(),
+ num_plugin_channels: num_plugin_channels.into(),
+ guest_fb_segment_count: 1.into(),
+ options: VgpuBootloadOptions::zeroed().into(),
+ channel_mapping,
+ guest_fb_segment_phys_addr: Array::new(&[guest_fb_address])?,
+ guest_fb_segment_length: Array::new(&[guest_fb_length])?,
+ plugin_heap_phys_addr: plugin_heap_address.into(),
+ plugin_heap_length: plugin_heap_length.into(),
+ ctrl_buff_offset: ctrl_buffer_offset.into(),
+ init_task_log_offset: init_log_address.into(),
+ init_task_log_size: init_log_size.into(),
+ vgpu_task_log_offset: vgpu_log_address.into(),
+ vgpu_task_log_size: vgpu_log_size.into(),
+ kernel_log_offset: kernel_log_address.into(),
+ kernel_log_size: kernel_log_size.into(),
+ mig_rm_heap_phys_addr: 0.into(),
+ mig_rm_heap_length: 0.into(),
+ };
+
+ let mut encoder = Encoder::new();
+ request.encode(&mut encoder)?;
+ Ok(encoder.finish())
+}
+
// VGPU_MGMT_QUERY_PROPERTIES
nvkv_decode! {
diff --git a/drivers/gpu/nova-core/mm/vram.rs b/drivers/gpu/nova-core/mm/vram.rs
index 87b7ce7f2c93..4a4bd42c9f18 100644
--- a/drivers/gpu/nova-core/mm/vram.rs
+++ b/drivers/gpu/nova-core/mm/vram.rs
@@ -86,19 +86,16 @@ fn new(backing: Arc<VramBlock>, range: Range<u64>) -> Result<Self> {
}
/// Return the physical address of the first byte in this region.
- #[expect(dead_code)]
pub(crate) const fn address(&self) -> u64 {
self.address
}
/// Return the region size in bytes.
- #[expect(dead_code)]
pub(crate) const fn size(&self) -> u64 {
self.size
}
/// Return a checked subregion relative to this region.
- #[expect(dead_code)]
pub(crate) fn subregion(&self, range: Range<u64>) -> Result<Self> {
let size = range
.end
diff --git a/drivers/gpu/nova-core/vgpu/bootload.rs b/drivers/gpu/nova-core/vgpu/bootload.rs
new file mode 100644
index 000000000000..0d382af46158
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/bootload.rs
@@ -0,0 +1,162 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+use kernel::{
+ device,
+ prelude::*,
+ time::Delta,
+ transmute::AsBytes, //
+};
+
+use crate::{
+ driver::Bar0,
+ gsp::{
+ cmdq::Cmdq,
+ commands::{
+ encode_vgpu_bootload,
+ ChannelMapEntry,
+ FifoEngineList, //
+ },
+ },
+ vgpu::consts::gmc, //
+};
+
+use super::instance::{
+ Gfid,
+ VgpuInstance, //
+};
+
+/// Build the typed channel mapping from the GSP FIFO engine list.
+fn channel_mapping(
+ fifo_engine_list: &FifoEngineList,
+ chid_offset: u32,
+) -> Result<KVVec<ChannelMapEntry>> {
+ let mut mapping = KVVec::new();
+ for &gmc_id in &fifo_engine_list.gmc_ids[..fifo_engine_list.count] {
+ let engine_type = (gmc_id & 0xffff) as usize;
+ let index = gmc_id >> 16;
+ mapping.push(
+ ChannelMapEntry::new(engine_type, index, chid_offset)?,
+ GFP_KERNEL,
+ )?;
+ }
+ Ok(mapping)
+}
+
+/// Bootload the GSP vGPU plugin and wait for its BAR1 ready indication.
+pub(crate) fn bootload(
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ instance: &VgpuInstance<'_>,
+ fifo_engine_list: &FifoEngineList,
+) -> Result {
+ let fb = &instance.vram_slot.fbmem;
+ let mgmt = &instance.vram_slot.mgmt_heap;
+ let logs = instance.plugin_rpc.plugin_logs()?;
+
+ let payload = encode_vgpu_bootload(
+ instance.dbdf,
+ instance.gfid.0,
+ instance.vgpu_type.vgpu_type_id(),
+ instance.vm_pid,
+ u32::try_from(instance.chids.len()).map_err(|_| EOVERFLOW)?,
+ instance.num_plugin_channels,
+ channel_mapping(
+ fifo_engine_list,
+ u32::try_from(instance.chids.start).map_err(|_| EOVERFLOW)?,
+ )?,
+ fb.address(),
+ fb.size(),
+ mgmt.address(),
+ mgmt.size(),
+ 0,
+ logs.init().address(),
+ logs.init().size(),
+ logs.vgpu().address(),
+ logs.vgpu().size(),
+ logs.kernel().address(),
+ logs.kernel().size(),
+ )?;
+
+ dev_dbg!(
+ dev,
+ "bootload: gfid={} sending {} typed NVKV bytes\n",
+ instance.gfid.0,
+ payload.len() * size_of::<u64>(),
+ );
+
+ // BOOTLOAD completes synchronously. The receive path dispatches any RM RPC
+ // frames that arrive before matching the response by command and sequence.
+ let response = cmdq.send_gmc_and_receive_timeout(
+ bar,
+ gmc::BOOTLOAD,
+ AsBytes::as_bytes(payload.as_slice()),
+ 0,
+ Delta::from_secs(10),
+ )?;
+ if response.status != 0 {
+ return Err(EIO);
+ }
+
+ instance.plugin_rpc.wait_plugin_ready(dev)?;
+
+ dev_dbg!(dev, "bootload: gfid={} plugin ready\n", instance.gfid.0);
+ Ok(())
+}
+
+/// Shut down a vGPU plugin task and wait for its completion event.
+pub(crate) fn shutdown(
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ gfid: Gfid,
+) -> Result {
+ let payload = gfid.0.to_le_bytes();
+
+ cmdq.send_gmc_and_wait_event(
+ bar,
+ gmc::SHUTDOWN,
+ &payload,
+ Delta::from_secs(10),
+ |command_id, status, _sequence, payload_0, payload_1| {
+ if command_id != gmc::SHUTDOWN_COMPLETE
+ || !payload
+ .iter()
+ .copied()
+ .eq(Iterator::chain(payload_0.iter(), payload_1.iter())
+ .take(payload.len())
+ .copied())
+ {
+ return Ok(false);
+ }
+ if status != 0 {
+ return Err(EIO);
+ }
+ Ok(true)
+ },
+ |command_id, status, _sequence, _payload_0, _payload_1| {
+ dev_dbg!(
+ dev,
+ "shutdown: ignoring unrelated event command={:#x} status={:#x}\n",
+ command_id,
+ status,
+ );
+ Ok(())
+ },
+ )?;
+ dev_dbg!(dev, "shutdown: gfid={} stopped\n", gfid.0);
+ Ok(())
+}
+
+/// Release firmware resources after a plugin task has stopped.
+pub(crate) fn cleanup(
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ gfid: Gfid,
+) -> Result {
+ cmdq.send_gmc_and_check_status(bar, gmc::CLEANUP, &gfid.0.to_le_bytes())?;
+ dev_dbg!(dev, "cleanup: gfid={} done\n", gfid.0);
+ Ok(())
+}
diff --git a/drivers/gpu/nova-core/vgpu/consts.rs b/drivers/gpu/nova-core/vgpu/consts.rs
index 7ec577ec12f2..2ebbf2a0daa3 100644
--- a/drivers/gpu/nova-core/vgpu/consts.rs
+++ b/drivers/gpu/nova-core/vgpu/consts.rs
@@ -9,4 +9,12 @@ pub(crate) mod gmc {
bindings::GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_VGPU_PROPERTIES;
pub(crate) const VGPU_MGMT_QUERY_ASSIGNED_VF: u32 =
bindings::GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_ASSIGNED_VF_VGPU_TYPE;
+ pub(crate) const BOOTLOAD: u32 =
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_BOOTLOAD_GSP_VGPU_PLUGIN_TASK;
+ pub(crate) const SHUTDOWN: u32 =
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK;
+ pub(crate) const SHUTDOWN_COMPLETE: u32 =
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK_COMPLETE;
+ pub(crate) const CLEANUP: u32 =
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_CLEANUP_GSP_VGPU_PLUGIN_RESOURCES;
}
diff --git a/drivers/gpu/nova-core/vgpu/fw.rs b/drivers/gpu/nova-core/vgpu/fw.rs
index edfb0f984b6d..3cd77f0cd62e 100644
--- a/drivers/gpu/nova-core/vgpu/fw.rs
+++ b/drivers/gpu/nova-core/vgpu/fw.rs
@@ -1,2 +1,195 @@
// SPDX-License-Identifier: GPL-2.0
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+use kernel::prelude::*;
+
+use crate::{
+ gsp::vgpu_bindings as bindings,
+ mm::{
+ bar_user::{
+ Bar1Map,
+ BarUser, //
+ },
+ vram::VramRegion,
+ GpuMm, //
+ },
+};
+
+type RawControlRegion = bindings::VGPU_CPU_GSP_CTRL_BUFF_REGION;
+
+/// Physical VRAM regions containing the vGPU plugin logs.
+pub(crate) struct PluginLogRegions {
+ init: VramRegion,
+ vgpu: VramRegion,
+ kernel: VramRegion,
+}
+
+impl PluginLogRegions {
+ /// Return the init-task log region.
+ pub(crate) const fn init(&self) -> &VramRegion {
+ &self.init
+ }
+
+ /// Return the vGPU-task log region.
+ pub(crate) const fn vgpu(&self) -> &VramRegion {
+ &self.vgpu
+ }
+
+ /// Return the kernel-task log region.
+ pub(crate) const fn kernel(&self) -> &VramRegion {
+ &self.kernel
+ }
+}
+
+/// Take the next firmware-defined subregion from a communication buffer.
+fn take_region(region: &VramRegion, cursor: &mut u64, size: u32) -> Result<VramRegion> {
+ let end = cursor.checked_add(u64::from(size)).ok_or(EOVERFLOW)?;
+ let subregion = region.subregion(*cursor..end)?;
+ *cursor = end;
+ Ok(subregion)
+}
+
+/// BAR1 mapping and semantic regions of a vGPU CPU-GSP communication buffer.
+///
+/// The host and GSP plugin exchange control, response, message, migration,
+/// error, and diagnostic data through firmware-defined subregions of the
+/// management heap. Firmware accesses that VRAM directly; the host accesses
+/// the same storage through the owned BAR1 mapping.
+///
+/// The firmware bindings define each subregion's size and order, but are used
+/// only to describe the layout. Field accesses must use the BAR1 I/O accessors
+/// because bindgen does not preserve C `volatile` semantics. Keep this object
+/// alive while the plugin or a host reader can use the buffer, then consume it
+/// with [`Self::destroy`] after those users have stopped.
+pub(crate) struct CommBufferRegion<'gpu> {
+ map: Bar1Map<'gpu>,
+ control: VramRegion,
+ init_log: VramRegion,
+ vgpu_log: VramRegion,
+ kernel_log: VramRegion,
+}
+
+impl<'gpu> CommBufferRegion<'gpu> {
+ /// Map the communication portion of a plugin management heap.
+ pub(crate) fn new(
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ management_heap: &VramRegion,
+ ) -> Result<Self> {
+ let total_size = u64::from(bindings::VGPU_CPU_GSP_COMMUNICATION_BUFF_TOTAL_SIZE);
+ let region = management_heap.subregion(0..total_size)?;
+ let mut cursor = 0;
+
+ let control = take_region(
+ ®ion,
+ &mut cursor,
+ bindings::VGPU_CPU_GSP_CTRL_BUFF_REGION_SIZE,
+ )?;
+ take_region(
+ ®ion,
+ &mut cursor,
+ bindings::VGPU_CPU_GSP_RESPONSE_BUFF_REGION_SIZE,
+ )?;
+ take_region(
+ ®ion,
+ &mut cursor,
+ bindings::VGPU_CPU_GSP_MESSAGE_BUFF_REGION_SIZE,
+ )?;
+ take_region(
+ ®ion,
+ &mut cursor,
+ bindings::VGPU_CPU_GSP_MIGRATION_BUFF_REGION_SIZE,
+ )?;
+ take_region(
+ ®ion,
+ &mut cursor,
+ bindings::VGPU_CPU_GSP_ERROR_BUFF_REGION_SIZE,
+ )?;
+ let init_log = take_region(
+ ®ion,
+ &mut cursor,
+ bindings::VGPU_CPU_GSP_INIT_TASK_LOG_BUFF_REGION_SIZE,
+ )?;
+ let vgpu_log = take_region(
+ ®ion,
+ &mut cursor,
+ bindings::VGPU_CPU_GSP_VGPU_TASK_LOG_BUFF_REGION_SIZE,
+ )?;
+ let kernel_log = take_region(
+ ®ion,
+ &mut cursor,
+ bindings::VGPU_CPU_GSP_KERNEL_TASK_LOG_BUFF_REGION_SIZE,
+ )?;
+ take_region(
+ ®ion,
+ &mut cursor,
+ bindings::VGPU_CPU_GSP_GUEST_RPC_TRACE_BUFF_REGION_SIZE,
+ )?;
+
+ if cursor != total_size || control.size() != u64::try_from(size_of::<RawControlRegion>())? {
+ return Err(EINVAL);
+ }
+
+ let map = Bar1Map::new(bar_user, mm, region, true)?;
+
+ Ok(Self {
+ map,
+ control,
+ init_log,
+ vgpu_log,
+ kernel_log,
+ })
+ }
+
+ fn region_offset(&self, region: &VramRegion) -> Result<usize> {
+ let offset = region
+ .address()
+ .checked_sub(self.map.region().address())
+ .ok_or(EINVAL)?;
+ if offset.checked_add(region.size()).ok_or(EOVERFLOW)? > self.map.region().size() {
+ return Err(EINVAL);
+ }
+
+ usize::try_from(offset).map_err(|_| EOVERFLOW)
+ }
+
+ fn io_offset(&self, region: &VramRegion, field: usize, width: usize) -> Result<usize> {
+ let field_end = field.checked_add(width).ok_or(EOVERFLOW)?;
+ if u64::try_from(field_end).map_err(|_| EOVERFLOW)? > region.size() {
+ return Err(EINVAL);
+ }
+
+ self.region_offset(region)?
+ .checked_add(field)
+ .ok_or(EOVERFLOW)
+ }
+
+ fn read_u32(&self, region: &VramRegion, field: usize) -> Result<u32> {
+ self.map
+ .try_read32(self.io_offset(region, field, size_of::<u32>())?)
+ }
+
+ /// Return the physical regions occupied by the three plugin logs.
+ pub(crate) fn plugin_logs(&self) -> Result<PluginLogRegions> {
+ Ok(PluginLogRegions {
+ init: self.init_log.clone(),
+ vgpu: self.vgpu_log.clone(),
+ kernel: self.kernel_log.clone(),
+ })
+ }
+
+ /// Return whether firmware has published the plugin boot marker.
+ pub(crate) fn is_plugin_ready(&self) -> Result<bool> {
+ let value = self.read_u32(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.message_seq_num),
+ )?;
+
+ Ok(value == bindings::GSP_PLUGIN_BOOTLOADED)
+ }
+
+ /// Invalidate the PTEs and release the communication mapping.
+ pub(crate) fn destroy(self, bar_user: &BarUser<'gpu>, mm: &mut GpuMm<'_>) -> Result {
+ self.map.destroy(bar_user, mm)
+ }
+}
diff --git a/drivers/gpu/nova-core/vgpu/instance.rs b/drivers/gpu/nova-core/vgpu/instance.rs
index ed304945330a..834b647e254a 100644
--- a/drivers/gpu/nova-core/vgpu/instance.rs
+++ b/drivers/gpu/nova-core/vgpu/instance.rs
@@ -4,6 +4,7 @@
use core::num::NonZeroUsize;
use kernel::{
+ device,
prelude::*,
ptr::Alignment,
sizes::SizeConstants, //
@@ -17,12 +18,23 @@
commands::{
decode_vgpu_properties,
Dbdf,
+ FifoEngineList,
VgpuProperties, //
},
},
- mm::GpuMm,
+ mm::{
+ bar_user::BarUser,
+ GpuMm, //
+ },
vgpu::{
+ bootload::{
+ bootload,
+ cleanup,
+ shutdown, //
+ },
consts::gmc,
+ fw::CommBufferRegion,
+ plugin_rpc::PluginRpc,
vram::{
VgpuVramLayout,
VgpuVramSlot,
@@ -59,6 +71,10 @@ pub(crate) struct VgpuType {
}
impl VgpuType {
+ pub(crate) const fn vgpu_type_id(&self) -> u32 {
+ self.vgpu_type_id
+ }
+
fn from_properties(properties: &VgpuProperties) -> Self {
let mut name = [0; 64];
let name_len = properties.name.len().min(name.len());
@@ -99,6 +115,24 @@ pub(crate) struct VgpuInstance<'gpu> {
pub(crate) chids: ChannelIdReservation<'gpu>,
pub(crate) num_plugin_channels: u32,
pub(crate) vram_slot: VgpuVramSlot,
+ pub(crate) plugin_rpc: PluginRpc<'gpu>,
+}
+
+impl<'gpu> VgpuInstance<'gpu> {
+ /// Unmap the plugin communication buffer and return the slot release token.
+ fn unmap_and_take_slot(
+ self,
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ ) -> Result<VgpuVramSlot> {
+ let Self {
+ plugin_rpc,
+ vram_slot,
+ ..
+ } = self;
+ plugin_rpc.destroy(bar_user, mm)?;
+ Ok(vram_slot)
+ }
}
/// Identity and firmware profile used to allocate an instance.
@@ -166,10 +200,13 @@ fn release_vram_slot(&mut self, slot: VgpuVramSlot) {
allocator.release(slot);
}
- /// Allocate resources and register a new inactive vGPU instance.
+ /// Allocate resources, map the management communication region, and
+ /// register a new inactive vGPU instance.
pub(crate) fn allocate_instance(
&mut self,
- mm: &GpuMm<'_>,
+ dev: &device::Device<device::Bound>,
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
vgpu: &VgpuManager<'gpu>,
info: InstanceInfo,
) -> Result<Gfid> {
@@ -220,6 +257,21 @@ pub(crate) fn allocate_instance(
fb_align: vgpu.vmmu_segment_size().ok_or(ENODEV)?,
};
let vram_slot = self.alloc_vram_slot(mm, layout)?;
+ let comm = match CommBufferRegion::new(bar_user, mm, &vram_slot.mgmt_heap) {
+ Ok(comm) => comm,
+ Err(error) => {
+ // A failed page-table update may have installed a partial mapping without
+ // returning a handle that can unmap it. Keep the slot reserved so its backing
+ // VRAM cannot be reused while stale BAR1 PTEs may still reference it.
+ dev_err!(
+ dev,
+ "allocate_instance: retaining slot {} after BAR1 map error {:?}\n",
+ vram_slot.index(),
+ error,
+ );
+ return Err(error);
+ }
+ };
let instance = VgpuInstance {
gfid,
@@ -229,26 +281,40 @@ pub(crate) fn allocate_instance(
chids,
num_plugin_channels: 3,
vram_slot,
+ plugin_rpc: PluginRpc::new(comm),
};
match self.instances.push_within_capacity(instance) {
Ok(()) => Ok(gfid),
- Err(error) => {
- let VgpuInstance { vram_slot, .. } = error.0;
- self.release_vram_slot(vram_slot);
- Err(EIO)
- }
+ Err(error) => match error.0.unmap_and_take_slot(bar_user, mm) {
+ Ok(vram_slot) => {
+ self.release_vram_slot(vram_slot);
+ Err(EIO)
+ }
+ Err(error) => Err(error),
+ },
}
}
- /// Remove an instance and release its channel and VRAM reservations.
- pub(crate) fn destroy_instance(&mut self, gfid: Gfid) -> Result {
+ /// Shut down and remove an instance, then release its reservations.
+ pub(crate) fn destroy_instance(
+ &mut self,
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ gfid: Gfid,
+ ) -> Result {
let index = self
.instances
.iter()
.position(|instance| instance.gfid == gfid)
.ok_or(ENOENT)?;
+
+ shutdown(dev, cmdq, bar, gfid)?;
+ cleanup(dev, cmdq, bar, gfid)?;
let instance = self.instances.remove(index).map_err(|_| EIO)?;
- let VgpuInstance { vram_slot, .. } = instance;
+ let vram_slot = instance.unmap_and_take_slot(bar_user, mm)?;
self.release_vram_slot(vram_slot);
Ok(())
}
@@ -286,3 +352,15 @@ pub(crate) fn query_vgpu_type(cmdq: &Cmdq, bar: Bar0<'_>, type_id: u32) -> Resul
}
Ok(VgpuType::from_properties(&properties))
}
+
+/// Bootload the GSP plugin for an allocated instance.
+#[expect(dead_code)]
+pub(crate) fn activate_instance(
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ instance: &mut VgpuInstance<'_>,
+ fifo_engine_list: &FifoEngineList,
+) -> Result {
+ bootload(dev, cmdq, bar, instance, fifo_engine_list)
+}
diff --git a/drivers/gpu/nova-core/vgpu/mod.rs b/drivers/gpu/nova-core/vgpu/mod.rs
index 320230ddd1dd..1c67d7afbe56 100644
--- a/drivers/gpu/nova-core/vgpu/mod.rs
+++ b/drivers/gpu/nova-core/vgpu/mod.rs
@@ -3,8 +3,10 @@
use core::num::NonZero;
+pub(crate) mod bootload;
pub(crate) mod consts;
pub(crate) mod instance;
+pub(crate) mod plugin_rpc;
pub(crate) use self::instance::VgpuInstances;
diff --git a/drivers/gpu/nova-core/vgpu/plugin_rpc.rs b/drivers/gpu/nova-core/vgpu/plugin_rpc.rs
new file mode 100644
index 000000000000..d6077a468672
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/plugin_rpc.rs
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+use kernel::{
+ device,
+ prelude::*,
+ time::{
+ delay::fsleep,
+ Delta,
+ Instant,
+ Monotonic, //
+ },
+};
+
+use crate::{
+ mm::{
+ bar_user::BarUser,
+ GpuMm, //
+ },
+ vgpu::fw::{
+ CommBufferRegion,
+ PluginLogRegions, //
+ },
+};
+
+/// Host-side ready limit from `vmiopd_negotiate_cpu_gsp_version()` in
+/// `vmiop-vgpu.c`, which polls the same boot marker for 10 seconds.
+const PLUGIN_READY_TIMEOUT: Delta = Delta::from_secs(10);
+
+/// BAR1-backed channel used to communicate with the vGPU plugin.
+pub(crate) struct PluginRpc<'gpu> {
+ comm: CommBufferRegion<'gpu>,
+}
+
+impl<'gpu> PluginRpc<'gpu> {
+ pub(crate) fn new(comm: CommBufferRegion<'gpu>) -> Self {
+ Self { comm }
+ }
+
+ /// Return the physical regions occupied by the plugin logs.
+ pub(crate) fn plugin_logs(&self) -> Result<PluginLogRegions> {
+ self.comm.plugin_logs()
+ }
+
+ /// Poll the control buffer until the plugin publishes its boot marker.
+ pub(crate) fn wait_plugin_ready(&self, dev: &device::Device<device::Bound>) -> Result {
+ let start = Instant::<Monotonic>::now();
+
+ loop {
+ if self.comm.is_plugin_ready()? {
+ dev_dbg!(dev, "vGPU plugin ready after {:?}\n", start.elapsed());
+ return Ok(());
+ }
+ if start.elapsed() >= PLUGIN_READY_TIMEOUT {
+ return Err(ETIMEDOUT);
+ }
+ fsleep(Delta::from_millis(1));
+ }
+ }
+
+ /// Release the BAR1 mapping.
+ pub(crate) fn destroy(self, bar_user: &BarUser<'gpu>, mm: &mut GpuMm<'_>) -> Result {
+ self.comm.destroy(bar_user, mm)
+ }
+}
diff --git a/drivers/gpu/nova-core/vgpu/vram.rs b/drivers/gpu/nova-core/vgpu/vram.rs
index c646511b6fd6..bc7283abba29 100644
--- a/drivers/gpu/nova-core/vgpu/vram.rs
+++ b/drivers/gpu/nova-core/vgpu/vram.rs
@@ -3,8 +3,6 @@
//! VRAM slot allocation for vGPU instances.
-#![expect(dead_code)]
-
use kernel::{
bitmap::BitmapVec,
prelude::*,
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 08/13] gpu: nova-core: vgpu: implement PluginRpc channel and config params
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
` (6 preceding siblings ...)
2026-09-05 8:11 ` [PATCH 07/13] gpu: nova-core: vgpu: add vGPU bootload Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 09/13] gpu: nova-core: vgpu: scrub guest framebuffer memory with CeUtils Zhi Wang
` (4 subsequent siblings)
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
Extend PluginRpc from a boot-ready poller into the BAR1-backed RPC
channel used by the vGPU plugin.
Initialize the control and response regions, publish NVKV-encoded
messages through the message region, ring the VF doorbell, and match
responses by sequence number. Negotiate the protocol, send the VM
configuration, and enable bus mastering after the plugin starts.
Keep raw firmware message IDs in fw/commands.rs and retain ownership of
the communication mapping in PluginRpc so teardown can release it after
firmware cleanup.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/gsp/commands.rs | 2 +
drivers/gpu/nova-core/gsp/fw.rs | 6 +
drivers/gpu/nova-core/gsp/fw/commands.rs | 45 +++++
drivers/gpu/nova-core/mm/bar_user.rs | 11 ++
drivers/gpu/nova-core/vgpu/consts.rs | 7 +
drivers/gpu/nova-core/vgpu/fw.rs | 192 +++++++++++++++++++-
drivers/gpu/nova-core/vgpu/fw/commands.rs | 29 +++
drivers/gpu/nova-core/vgpu/instance.rs | 30 +++-
drivers/gpu/nova-core/vgpu/plugin_rpc.rs | 209 +++++++++++++++++++++-
9 files changed, 519 insertions(+), 12 deletions(-)
create mode 100644 drivers/gpu/nova-core/vgpu/fw/commands.rs
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 481ec8e221ce..fbbff257300f 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -54,6 +54,8 @@
};
pub(crate) use fw::commands::{
+ encode_plugin_config_params,
+ encode_plugin_set_bme,
encode_vgpu_bootload,
ChannelMapEntry,
Dbdf,
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index d68b790533af..686120224d0f 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -14,15 +14,21 @@ pub(crate) mod vgpu_bindings {
GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK,
GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK_COMPLETE,
GSP_PLUGIN_BOOTLOADED,
+ MESSAGE_NV_VGPU_CPU_RPC_MSG_RESET,
+ MESSAGE_NV_VGPU_CPU_RPC_MSG_SETUP_CONFIG_PARAMS_AND_INIT,
+ MESSAGE_NV_VGPU_CPU_RPC_MSG_UPDATE_BME_STATE,
+ MESSAGE_NV_VGPU_CPU_RPC_MSG_VERSION_NEGOTIATION,
VGPU_CPU_GSP_COMMUNICATION_BUFF_TOTAL_SIZE,
VGPU_CPU_GSP_CTRL_BUFF_REGION,
VGPU_CPU_GSP_CTRL_BUFF_REGION_SIZE,
+ VGPU_CPU_GSP_CTRL_BUFF_VERSION,
VGPU_CPU_GSP_ERROR_BUFF_REGION_SIZE,
VGPU_CPU_GSP_GUEST_RPC_TRACE_BUFF_REGION_SIZE,
VGPU_CPU_GSP_INIT_TASK_LOG_BUFF_REGION_SIZE,
VGPU_CPU_GSP_KERNEL_TASK_LOG_BUFF_REGION_SIZE,
VGPU_CPU_GSP_MESSAGE_BUFF_REGION_SIZE,
VGPU_CPU_GSP_MIGRATION_BUFF_REGION_SIZE,
+ VGPU_CPU_GSP_RESPONSE_BUFF_REGION,
VGPU_CPU_GSP_RESPONSE_BUFF_REGION_SIZE,
VGPU_CPU_GSP_VGPU_TASK_LOG_BUFF_REGION_SIZE, //
};
diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs
index fe4a7af88ec7..d7d52fa33d29 100644
--- a/drivers/gpu/nova-core/gsp/fw/commands.rs
+++ b/drivers/gpu/nova-core/gsp/fw/commands.rs
@@ -784,6 +784,40 @@ impl PluginConfigParamsRequest {
const FEATURE_FLAGS_KEY: KeyId = 0x0030;
}
+/// Encodes plugin configuration parameters using the typed NVKV schema.
+pub(crate) fn encode_plugin_config_params(
+ uuid: [u8; 16],
+ dbdf: Dbdf,
+ vgpu_type: u32,
+ vm_pid: u32,
+ num_channels: u32,
+ num_plugin_channels: u32,
+) -> Result<KVVec<u64>> {
+ let request = PluginConfigParamsRequest {
+ uuid: uuid.into(),
+ dbdf: dbdf.into(),
+ dev_inst: 0.into(),
+ vgpu_type: vgpu_type.into(),
+ vm_pid: vm_pid.into(),
+ swizz_id: SwizzId::WHOLE_GPU.into(),
+ num_channels: num_channels.into(),
+ num_plugin_channels: num_plugin_channels.into(),
+ vmm_cap: 0.into(),
+ migration_feature: MigrationFeature::KVM.into(),
+ hypervisor_type: HypervisorType::Unknown.into(),
+ cpu_arch: CpuArch::X86_64.into(),
+ page_size: 4096.into(),
+ feature_flags: FeatureFlags::zeroed()
+ .with_enable_uvm(true)
+ .with_vmm_migration(true)
+ .into(),
+ };
+
+ let mut encoder = Encoder::new();
+ request.encode(&mut encoder)?;
+ Ok(encoder.finish())
+}
+
// UPDATE_BME_STATE
nvkv_encode! {
@@ -798,6 +832,17 @@ impl PluginSetBmeRequest {
const BME_ENABLE_KEY: KeyId = 0x0100;
}
+/// Encodes a plugin BME state update using the typed NVKV schema.
+pub(crate) fn encode_plugin_set_bme(enable: bool) -> Result<KVVec<u64>> {
+ let request = PluginSetBmeRequest {
+ bme_enable: enable.into(),
+ };
+
+ let mut encoder = Encoder::new();
+ request.encode(&mut encoder)?;
+ Ok(encoder.finish())
+}
+
#[kunit_tests(nova_core_fw_commands)]
mod tests {
use crate::gsp::nvkv::{
diff --git a/drivers/gpu/nova-core/mm/bar_user.rs b/drivers/gpu/nova-core/mm/bar_user.rs
index adc23ac6d467..158160fbf352 100644
--- a/drivers/gpu/nova-core/mm/bar_user.rs
+++ b/drivers/gpu/nova-core/mm/bar_user.rs
@@ -141,6 +141,12 @@ pub(crate) fn try_read32(&self, offset: usize) -> Result<u32> {
self.bar_user.bar1.try_read32(off)
}
+ /// Write an 8-bit value at the given offset.
+ pub(crate) fn try_write8(&self, value: u8, offset: usize) -> Result {
+ let off = self.bar_offset(offset)?;
+ self.bar_user.bar1.try_write8(value, off)
+ }
+
/// Write a 32-bit value at the given offset.
pub(crate) fn try_write32(&self, value: u32, offset: usize) -> Result {
let off = self.bar_offset(offset)?;
@@ -270,6 +276,11 @@ pub(crate) fn try_read32(&self, offset: usize) -> Result<u32> {
.try_read32(self.bar_offset(offset, size_of::<u32>())?)
}
+ pub(crate) fn try_write8(&self, value: u8, offset: usize) -> Result {
+ self.bar1
+ .try_write8(value, self.bar_offset(offset, size_of::<u8>())?)
+ }
+
pub(crate) fn try_write32(&self, value: u32, offset: usize) -> Result {
self.bar1
.try_write32(value, self.bar_offset(offset, size_of::<u32>())?)
diff --git a/drivers/gpu/nova-core/vgpu/consts.rs b/drivers/gpu/nova-core/vgpu/consts.rs
index 2ebbf2a0daa3..13aabefa4ecf 100644
--- a/drivers/gpu/nova-core/vgpu/consts.rs
+++ b/drivers/gpu/nova-core/vgpu/consts.rs
@@ -18,3 +18,10 @@ pub(crate) mod gmc {
pub(crate) const CLEANUP: u32 =
bindings::GMCAPI_COMMANDS_GMCAPI_CMD_CLEANUP_GSP_VGPU_PLUGIN_RESOURCES;
}
+
+/// vGPU plugin RPC values not provided by the firmware bindings.
+pub(crate) mod plugin_rpc {
+ pub(crate) const DOORBELL_STRIDE: u32 = 32;
+ pub(crate) const DOORBELL_VECTOR: u32 = 17;
+ pub(crate) const NV_VIRTUAL_FUNCTION_PRIV_DOORBELL: usize = 0xb8_0000 + 0x2200;
+}
diff --git a/drivers/gpu/nova-core/vgpu/fw.rs b/drivers/gpu/nova-core/vgpu/fw.rs
index 3cd77f0cd62e..db6f535998a9 100644
--- a/drivers/gpu/nova-core/vgpu/fw.rs
+++ b/drivers/gpu/nova-core/vgpu/fw.rs
@@ -1,6 +1,13 @@
// SPDX-License-Identifier: GPL-2.0
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+mod commands;
+
+pub(crate) use commands::{
+ RpcMessage,
+ RpcResponse, //
+};
+
use kernel::prelude::*;
use crate::{
@@ -16,6 +23,7 @@
};
type RawControlRegion = bindings::VGPU_CPU_GSP_CTRL_BUFF_REGION;
+type RawResponseRegion = bindings::VGPU_CPU_GSP_RESPONSE_BUFF_REGION;
/// Physical VRAM regions containing the vGPU plugin logs.
pub(crate) struct PluginLogRegions {
@@ -64,9 +72,14 @@ fn take_region(region: &VramRegion, cursor: &mut u64, size: u32) -> Result<VramR
pub(crate) struct CommBufferRegion<'gpu> {
map: Bar1Map<'gpu>,
control: VramRegion,
+ response: VramRegion,
+ message: VramRegion,
+ migration: VramRegion,
+ error: VramRegion,
init_log: VramRegion,
vgpu_log: VramRegion,
kernel_log: VramRegion,
+ guest_trace: VramRegion,
}
impl<'gpu> CommBufferRegion<'gpu> {
@@ -85,22 +98,22 @@ pub(crate) fn new(
&mut cursor,
bindings::VGPU_CPU_GSP_CTRL_BUFF_REGION_SIZE,
)?;
- take_region(
+ let response = take_region(
®ion,
&mut cursor,
bindings::VGPU_CPU_GSP_RESPONSE_BUFF_REGION_SIZE,
)?;
- take_region(
+ let message = take_region(
®ion,
&mut cursor,
bindings::VGPU_CPU_GSP_MESSAGE_BUFF_REGION_SIZE,
)?;
- take_region(
+ let migration = take_region(
®ion,
&mut cursor,
bindings::VGPU_CPU_GSP_MIGRATION_BUFF_REGION_SIZE,
)?;
- take_region(
+ let error = take_region(
®ion,
&mut cursor,
bindings::VGPU_CPU_GSP_ERROR_BUFF_REGION_SIZE,
@@ -120,13 +133,16 @@ pub(crate) fn new(
&mut cursor,
bindings::VGPU_CPU_GSP_KERNEL_TASK_LOG_BUFF_REGION_SIZE,
)?;
- take_region(
+ let guest_trace = take_region(
®ion,
&mut cursor,
bindings::VGPU_CPU_GSP_GUEST_RPC_TRACE_BUFF_REGION_SIZE,
)?;
- if cursor != total_size || control.size() != u64::try_from(size_of::<RawControlRegion>())? {
+ if cursor != total_size
+ || control.size() != u64::try_from(size_of::<RawControlRegion>())?
+ || response.size() != u64::try_from(size_of::<RawResponseRegion>())?
+ {
return Err(EINVAL);
}
@@ -135,9 +151,14 @@ pub(crate) fn new(
Ok(Self {
map,
control,
+ response,
+ message,
+ migration,
+ error,
init_log,
vgpu_log,
kernel_log,
+ guest_trace,
})
}
@@ -169,6 +190,21 @@ fn read_u32(&self, region: &VramRegion, field: usize) -> Result<u32> {
.try_read32(self.io_offset(region, field, size_of::<u32>())?)
}
+ fn write_u8(&self, region: &VramRegion, field: usize, value: u8) -> Result {
+ self.map
+ .try_write8(value, self.io_offset(region, field, size_of::<u8>())?)
+ }
+
+ fn write_u32(&self, region: &VramRegion, field: usize, value: u32) -> Result {
+ self.map
+ .try_write32(value, self.io_offset(region, field, size_of::<u32>())?)
+ }
+
+ fn write_u64(&self, region: &VramRegion, field: usize, value: u64) -> Result {
+ self.map
+ .try_write64(value, self.io_offset(region, field, size_of::<u64>())?)
+ }
+
/// Return the physical regions occupied by the three plugin logs.
pub(crate) fn plugin_logs(&self) -> Result<PluginLogRegions> {
Ok(PluginLogRegions {
@@ -188,6 +224,150 @@ pub(crate) fn is_plugin_ready(&self) -> Result<bool> {
Ok(value == bindings::GSP_PLUGIN_BOOTLOADED)
}
+ /// Initialize the shared control and response buffers for plugin RPC.
+ pub(crate) fn initialize(&self) -> Result {
+ self.write_u64(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.response_buff_offset),
+ u64::try_from(self.region_offset(&self.response)?)?,
+ )?;
+ self.write_u64(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.message_buff_offset),
+ u64::try_from(self.region_offset(&self.message)?)?,
+ )?;
+ self.write_u64(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.migration_buff_offset),
+ u64::try_from(self.region_offset(&self.migration)?)?,
+ )?;
+ self.write_u64(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.error_buff_offset),
+ u64::try_from(self.region_offset(&self.error)?)?,
+ )?;
+ self.write_u64(
+ &self.control,
+ core::mem::offset_of!(
+ RawControlRegion,
+ __bindgen_anon_1.guest_rpc_trace_buff_offset
+ ),
+ u64::try_from(self.region_offset(&self.guest_trace)?)?,
+ )?;
+ self.write_u32(
+ &self.control,
+ core::mem::offset_of!(
+ RawControlRegion,
+ __bindgen_anon_1.migration_buf_cpu_access_offset
+ ),
+ 0,
+ )?;
+ self.write_u8(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.is_migration_in_progress),
+ 0,
+ )?;
+ self.write_u32(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.error_buff_cpu_get_idx),
+ 0,
+ )?;
+ self.write_u32(
+ &self.control,
+ core::mem::offset_of!(
+ RawControlRegion,
+ __bindgen_anon_1.guest_rpc_trace_buff_cpu_get_idx
+ ),
+ 0,
+ )?;
+ self.write_u32(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.attached_vgpu_count),
+ 1,
+ )?;
+ self.write_u8(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.is_gr_init_done),
+ 0,
+ )?;
+
+ // The heap is not guaranteed to have been zeroed. Clear both sides'
+ // sequence state before publishing the control-buffer version.
+ self.write_u32(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.message_type),
+ 0,
+ )?;
+ self.write_u32(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.message_seq_num),
+ 0,
+ )?;
+ self.write_u32(
+ &self.response,
+ core::mem::offset_of!(
+ RawResponseRegion,
+ __bindgen_anon_1.message_seq_num_processed
+ ),
+ 0,
+ )?;
+ self.write_u32(
+ &self.response,
+ core::mem::offset_of!(RawResponseRegion, __bindgen_anon_1.result_code),
+ 0,
+ )?;
+ self.write_u32(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.version),
+ bindings::VGPU_CPU_GSP_CTRL_BUFF_VERSION,
+ )
+ }
+
+ /// Copy and publish one RPC request to firmware.
+ pub(crate) fn submit(&self, message: RpcMessage, sequence: u32, data: &[u8]) -> Result {
+ if u64::try_from(data.len()).map_err(|_| EOVERFLOW)? > self.message.size() {
+ return Err(E2BIG);
+ }
+
+ for (index, chunk) in data.chunks(size_of::<u32>()).enumerate() {
+ let mut bytes = [0u8; size_of::<u32>()];
+ bytes[..chunk.len()].copy_from_slice(chunk);
+ let field = index.checked_mul(size_of::<u32>()).ok_or(EOVERFLOW)?;
+ self.write_u32(&self.message, field, u32::from_le_bytes(bytes))?;
+ }
+
+ self.write_u32(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.message_type),
+ message as u32,
+ )?;
+ self.write_u32(
+ &self.control,
+ core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.message_seq_num),
+ sequence,
+ )
+ }
+
+ /// Read firmware's response for an expected RPC sequence.
+ pub(crate) fn response(&self, expected_sequence: u32) -> Result<RpcResponse> {
+ let sequence = self.read_u32(
+ &self.response,
+ core::mem::offset_of!(
+ RawResponseRegion,
+ __bindgen_anon_1.message_seq_num_processed
+ ),
+ )?;
+ if sequence != expected_sequence {
+ return Ok(RpcResponse::Pending { sequence });
+ }
+
+ let status = self.read_u32(
+ &self.response,
+ core::mem::offset_of!(RawResponseRegion, __bindgen_anon_1.result_code),
+ )?;
+ Ok(RpcResponse::Complete { status })
+ }
+
/// Invalidate the PTEs and release the communication mapping.
pub(crate) fn destroy(self, bar_user: &BarUser<'gpu>, mm: &mut GpuMm<'_>) -> Result {
self.map.destroy(bar_user, mm)
diff --git a/drivers/gpu/nova-core/vgpu/fw/commands.rs b/drivers/gpu/nova-core/vgpu/fw/commands.rs
new file mode 100644
index 000000000000..8527e2c430bf
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/fw/commands.rs
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+use crate::gsp::vgpu_bindings as bindings;
+
+/// State observed in the response buffer for an expected RPC sequence.
+pub(crate) enum RpcResponse {
+ /// Firmware has not completed the expected sequence.
+ Pending {
+ /// Last sequence completed by firmware.
+ sequence: u32,
+ },
+ /// Firmware has completed the expected sequence.
+ Complete {
+ /// Firmware result code.
+ status: u32,
+ },
+}
+
+/// Message types supported by the nova-core plugin RPC channel.
+#[derive(Clone, Copy)]
+#[repr(u32)]
+pub(crate) enum RpcMessage {
+ VersionNegotiation = bindings::MESSAGE_NV_VGPU_CPU_RPC_MSG_VERSION_NEGOTIATION,
+ SetupConfigParamsAndInit = bindings::MESSAGE_NV_VGPU_CPU_RPC_MSG_SETUP_CONFIG_PARAMS_AND_INIT,
+ #[expect(dead_code)]
+ Reset = bindings::MESSAGE_NV_VGPU_CPU_RPC_MSG_RESET,
+ UpdateBmeState = bindings::MESSAGE_NV_VGPU_CPU_RPC_MSG_UPDATE_BME_STATE,
+}
diff --git a/drivers/gpu/nova-core/vgpu/instance.rs b/drivers/gpu/nova-core/vgpu/instance.rs
index 834b647e254a..161b49d93de5 100644
--- a/drivers/gpu/nova-core/vgpu/instance.rs
+++ b/drivers/gpu/nova-core/vgpu/instance.rs
@@ -34,7 +34,10 @@
},
consts::gmc,
fw::CommBufferRegion,
- plugin_rpc::PluginRpc,
+ plugin_rpc::{
+ PluginConfigParams,
+ PluginRpc, //
+ },
vram::{
VgpuVramLayout,
VgpuVramSlot,
@@ -353,7 +356,11 @@ pub(crate) fn query_vgpu_type(cmdq: &Cmdq, bar: Bar0<'_>, type_id: u32) -> Resul
Ok(VgpuType::from_properties(&properties))
}
-/// Bootload the GSP plugin for an allocated instance.
+/// Start the vGPU plugin and establish its RPC channel.
+///
+/// Ask GSP to create the plugin task, wait for its BAR1 ready marker,
+/// initialize the shared RPC buffers, negotiate the protocol, send the
+/// instance configuration, and enable bus mastering.
#[expect(dead_code)]
pub(crate) fn activate_instance(
dev: &device::Device<device::Bound>,
@@ -362,5 +369,22 @@ pub(crate) fn activate_instance(
instance: &mut VgpuInstance<'_>,
fifo_engine_list: &FifoEngineList,
) -> Result {
- bootload(dev, cmdq, bar, instance, fifo_engine_list)
+ bootload(dev, cmdq, bar, instance, fifo_engine_list)?;
+
+ let params = PluginConfigParams::new(
+ [0; 16],
+ instance.dbdf,
+ instance.vgpu_type.vgpu_type_id,
+ instance.vm_pid,
+ u32::try_from(instance.chids.len()).map_err(|_| EOVERFLOW)?,
+ instance.num_plugin_channels,
+ );
+ let gfid = instance.gfid;
+ let rpc = &mut instance.plugin_rpc;
+ rpc.init_rpc()?;
+ rpc.negotiate_rpc_version(dev, bar, gfid)?;
+ rpc.send_config_params(dev, bar, gfid, ¶ms)?;
+ rpc.set_bme(dev, bar, gfid, true)?;
+
+ Ok(())
}
diff --git a/drivers/gpu/nova-core/vgpu/plugin_rpc.rs b/drivers/gpu/nova-core/vgpu/plugin_rpc.rs
index d6077a468672..f1fa6fe238fb 100644
--- a/drivers/gpu/nova-core/vgpu/plugin_rpc.rs
+++ b/drivers/gpu/nova-core/vgpu/plugin_rpc.rs
@@ -3,6 +3,7 @@
use kernel::{
device,
+ io::Io,
prelude::*,
time::{
delay::fsleep,
@@ -10,31 +11,80 @@
Instant,
Monotonic, //
},
+ transmute::AsBytes, //
};
use crate::{
+ driver::Bar0,
+ gsp::commands::{
+ encode_plugin_config_params,
+ encode_plugin_set_bme,
+ Dbdf, //
+ },
mm::{
bar_user::BarUser,
GpuMm, //
},
vgpu::fw::{
CommBufferRegion,
- PluginLogRegions, //
+ PluginLogRegions,
+ RpcMessage,
+ RpcResponse, //
},
};
+use super::{
+ consts::plugin_rpc as consts,
+ instance::Gfid, //
+};
+
/// Host-side ready limit from `vmiopd_negotiate_cpu_gsp_version()` in
/// `vmiop-vgpu.c`, which polls the same boot marker for 10 seconds.
const PLUGIN_READY_TIMEOUT: Delta = Delta::from_secs(10);
-/// BAR1-backed channel used to communicate with the vGPU plugin.
+/// Values sent in the plugin's setup-configuration RPC.
+pub(crate) struct PluginConfigParams {
+ uuid: [u8; 16],
+ dbdf: Dbdf,
+ vgpu_type: u32,
+ vm_pid: u32,
+ num_channels: u32,
+ num_plugin_channels: u32,
+}
+
+impl PluginConfigParams {
+ pub(crate) const fn new(
+ uuid: [u8; 16],
+ dbdf: Dbdf,
+ vgpu_type: u32,
+ vm_pid: u32,
+ num_channels: u32,
+ num_plugin_channels: u32,
+ ) -> Self {
+ Self {
+ uuid,
+ dbdf,
+ vgpu_type,
+ vm_pid,
+ num_channels,
+ num_plugin_channels,
+ }
+ }
+}
+
+/// BAR1-backed channel used to communicate with one vGPU plugin.
pub(crate) struct PluginRpc<'gpu> {
comm: CommBufferRegion<'gpu>,
+ message_sequence: u32,
}
impl<'gpu> PluginRpc<'gpu> {
+ /// Create a channel over the mapped communication buffer.
pub(crate) fn new(comm: CommBufferRegion<'gpu>) -> Self {
- Self { comm }
+ Self {
+ comm,
+ message_sequence: 0,
+ }
}
/// Return the physical regions occupied by the plugin logs.
@@ -58,8 +108,161 @@ pub(crate) fn wait_plugin_ready(&self, dev: &device::Device<device::Bound>) -> R
}
}
+ /// Initialize the control and response buffers for the first RPC.
+ pub(crate) fn init_rpc(&mut self) -> Result {
+ self.comm.initialize()?;
+ self.message_sequence = 0;
+ Ok(())
+ }
+
+ fn next_sequence(&self) -> u32 {
+ let sequence = self.message_sequence.wrapping_add(1);
+ if sequence == 0 {
+ 1
+ } else {
+ sequence
+ }
+ }
+
+ /// Write one RPC message, ring the VF doorbell, and wait for its response.
+ pub(crate) fn rpc_call(
+ &mut self,
+ dev: &device::Device<device::Bound>,
+ bar0: Bar0<'_>,
+ gfid: Gfid,
+ message_type: RpcMessage,
+ data: &[u8],
+ ) -> Result {
+ let sequence = self.next_sequence();
+ self.comm.submit(message_type, sequence, data)?;
+ self.message_sequence = sequence;
+
+ dev_dbg!(
+ dev,
+ "vGPU RPC: gfid={} type={} bytes={} sequence={}\n",
+ gfid.0,
+ message_type as u32,
+ data.len(),
+ sequence,
+ );
+
+ ring_doorbell(bar0, gfid)?;
+ self.wait_response(dev, sequence)
+ }
+
+ fn wait_response(&self, dev: &device::Device<device::Bound>, expected_sequence: u32) -> Result {
+ let start = Instant::<Monotonic>::now();
+ let timeout = Delta::from_secs(120);
+
+ loop {
+ match self.comm.response(expected_sequence)? {
+ RpcResponse::Complete { status } => {
+ if status != 0 {
+ dev_dbg!(
+ dev,
+ "vGPU RPC: sequence {} failed with status {}\n",
+ expected_sequence,
+ status,
+ );
+ return Err(EIO);
+ }
+
+ dev_dbg!(
+ dev,
+ "vGPU RPC: sequence {} completed after {:?}\n",
+ expected_sequence,
+ start.elapsed(),
+ );
+ return Ok(());
+ }
+ RpcResponse::Pending { sequence } => {
+ if start.elapsed() >= timeout {
+ dev_dbg!(
+ dev,
+ "vGPU RPC: sequence {} timed out; last response was {}\n",
+ expected_sequence,
+ sequence,
+ );
+ return Err(ETIMEDOUT);
+ }
+ }
+ }
+ fsleep(Delta::from_millis(1));
+ }
+ }
+
+ /// Negotiate the plugin RPC protocol version.
+ pub(crate) fn negotiate_rpc_version(
+ &mut self,
+ dev: &device::Device<device::Bound>,
+ bar0: Bar0<'_>,
+ gfid: Gfid,
+ ) -> Result {
+ self.rpc_call(dev, bar0, gfid, RpcMessage::VersionNegotiation, &[])
+ }
+
+ /// Send the NVKV-encoded v2 configuration message.
+ pub(crate) fn send_config_params(
+ &mut self,
+ dev: &device::Device<device::Bound>,
+ bar0: Bar0<'_>,
+ gfid: Gfid,
+ params: &PluginConfigParams,
+ ) -> Result {
+ let encoded = encode_plugin_config_params(
+ params.uuid,
+ params.dbdf,
+ params.vgpu_type,
+ params.vm_pid,
+ params.num_channels,
+ params.num_plugin_channels,
+ )?;
+ let payload = nvkv_rpc_payload(&encoded)?;
+
+ self.rpc_call(
+ dev,
+ bar0,
+ gfid,
+ RpcMessage::SetupConfigParamsAndInit,
+ &payload,
+ )
+ }
+
+ /// Send a Bus Master Enable state update.
+ pub(crate) fn set_bme(
+ &mut self,
+ dev: &device::Device<device::Bound>,
+ bar0: Bar0<'_>,
+ gfid: Gfid,
+ enable: bool,
+ ) -> Result {
+ let encoded = encode_plugin_set_bme(enable)?;
+ let payload = nvkv_rpc_payload(&encoded)?;
+
+ self.rpc_call(dev, bar0, gfid, RpcMessage::UpdateBmeState, &payload)
+ }
+
/// Release the BAR1 mapping.
pub(crate) fn destroy(self, bar_user: &BarUser<'gpu>, mm: &mut GpuMm<'_>) -> Result {
self.comm.destroy(bar_user, mm)
}
}
+
+fn nvkv_rpc_payload(encoded: &[u64]) -> Result<KVec<u8>> {
+ let word_count = u64::try_from(encoded.len()).map_err(|_| EOVERFLOW)?;
+ let mut payload = KVec::new();
+ payload.extend_from_slice(&word_count.to_le_bytes(), GFP_KERNEL)?;
+ payload.extend_from_slice(AsBytes::as_bytes(encoded), GFP_KERNEL)?;
+ Ok(payload)
+}
+
+fn ring_doorbell(bar0: Bar0<'_>, gfid: Gfid) -> Result {
+ let value = gfid
+ .0
+ .checked_mul(consts::DOORBELL_STRIDE)
+ .and_then(|value| value.checked_add(consts::DOORBELL_VECTOR))
+ .ok_or(EOVERFLOW)?;
+ bar0.try_write32(value, consts::NV_VIRTUAL_FUNCTION_PRIV_DOORBELL)?;
+ bar0.try_read32(consts::NV_VIRTUAL_FUNCTION_PRIV_DOORBELL)?;
+ Ok(())
+}
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 09/13] gpu: nova-core: vgpu: scrub guest framebuffer memory with CeUtils
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
` (7 preceding siblings ...)
2026-09-05 8:11 ` [PATCH 08/13] gpu: nova-core: vgpu: implement PluginRpc channel and config params Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 10/13] gpu: nova-core: vgpu: export plugin log buffers via debugfs Zhi Wang
` (3 subsequent siblings)
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
A vGPU framebuffer can retain guest data when an instance is reused.
Add a per-instance Copy Engine utility, CeUtils, that asks GSP-RM to
scrub the framebuffer and verifies completion through a hardware
semaphore page.
Scrub a framebuffer immediately after allocating it and after shutting
an instance down, before returning its VRAM to the allocator. Reserve
the last channel ID in each instance range for CeUtils and report the
remaining channel count to the plugin.
If firmware ownership or scrub completion cannot be established, retain
the affected channel IDs and VRAM instead of allowing another instance
to reuse them.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/gsp/fw.rs | 4 +
.../gpu/nova-core/gsp/fw/r000_00/bindings.rs | 1 +
drivers/gpu/nova-core/vgpu/consts.rs | 6 +
drivers/gpu/nova-core/vgpu/instance.rs | 141 +++++-
drivers/gpu/nova-core/vgpu/mod.rs | 1 +
drivers/gpu/nova-core/vgpu/scrubber.rs | 474 ++++++++++++++++++
6 files changed, 615 insertions(+), 12 deletions(-)
create mode 100644 drivers/gpu/nova-core/vgpu/scrubber.rs
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index 686120224d0f..e6c0fac55bad 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -13,11 +13,15 @@ pub(crate) mod vgpu_bindings {
GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_VGPU_PROPERTIES,
GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK,
GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK_COMPLETE,
+ GMCAPI_COMMANDS_GMCAPI_CMD_VGPU_MGR_ALLOC_GSP_CEUTILS,
+ GMCAPI_COMMANDS_GMCAPI_CMD_VGPU_MGR_FREE_GSP_CEUTILS,
+ GMCAPI_COMMANDS_GMCAPI_CMD_VGPU_MGR_SCRUB_GUEST_FB,
GSP_PLUGIN_BOOTLOADED,
MESSAGE_NV_VGPU_CPU_RPC_MSG_RESET,
MESSAGE_NV_VGPU_CPU_RPC_MSG_SETUP_CONFIG_PARAMS_AND_INIT,
MESSAGE_NV_VGPU_CPU_RPC_MSG_UPDATE_BME_STATE,
MESSAGE_NV_VGPU_CPU_RPC_MSG_VERSION_NEGOTIATION,
+ NV_ADDR_FBMEM,
VGPU_CPU_GSP_COMMUNICATION_BUFF_TOTAL_SIZE,
VGPU_CPU_GSP_CTRL_BUFF_REGION,
VGPU_CPU_GSP_CTRL_BUFF_REGION_SIZE,
diff --git a/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs b/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
index dcb44a403469..01e84dfc4b88 100644
--- a/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
+++ b/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
@@ -858,6 +858,7 @@ pub struct rpc_unloading_guest_driver_v1F_07 {
pub __bindgen_padding_0: [u8; 2usize],
pub newLevel: u32_,
}
+pub const NV_ADDR_FBMEM: u32 = 2;
pub const GSP_PLUGIN_BOOTLOADED: u32 = 1315261039;
pub const VGPU_CPU_GSP_CTRL_BUFF_VERSION: u32 = 2;
pub const VGPU_CPU_GSP_CTRL_BUFF_REGION_SIZE: u32 = 4096;
diff --git a/drivers/gpu/nova-core/vgpu/consts.rs b/drivers/gpu/nova-core/vgpu/consts.rs
index 13aabefa4ecf..3b3e8b239ac3 100644
--- a/drivers/gpu/nova-core/vgpu/consts.rs
+++ b/drivers/gpu/nova-core/vgpu/consts.rs
@@ -17,6 +17,12 @@ pub(crate) mod gmc {
bindings::GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK_COMPLETE;
pub(crate) const CLEANUP: u32 =
bindings::GMCAPI_COMMANDS_GMCAPI_CMD_CLEANUP_GSP_VGPU_PLUGIN_RESOURCES;
+ pub(crate) const SCRUB_GUEST_FB: u32 =
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_VGPU_MGR_SCRUB_GUEST_FB;
+ pub(crate) const ALLOC_GSP_CEUTILS: u32 =
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_VGPU_MGR_ALLOC_GSP_CEUTILS;
+ pub(crate) const FREE_GSP_CEUTILS: u32 =
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_VGPU_MGR_FREE_GSP_CEUTILS;
}
/// vGPU plugin RPC values not provided by the firmware bindings.
diff --git a/drivers/gpu/nova-core/vgpu/instance.rs b/drivers/gpu/nova-core/vgpu/instance.rs
index 161b49d93de5..d6938cc65bf9 100644
--- a/drivers/gpu/nova-core/vgpu/instance.rs
+++ b/drivers/gpu/nova-core/vgpu/instance.rs
@@ -38,6 +38,10 @@
PluginConfigParams,
PluginRpc, //
},
+ scrubber::{
+ CeUtils,
+ CeUtilsAllocError, //
+ },
vram::{
VgpuVramLayout,
VgpuVramSlot,
@@ -117,11 +121,22 @@ pub(crate) struct VgpuInstance<'gpu> {
pub(crate) vm_pid: u32,
pub(crate) chids: ChannelIdReservation<'gpu>,
pub(crate) num_plugin_channels: u32,
+ ceutils: CeUtils,
pub(crate) vram_slot: VgpuVramSlot,
pub(crate) plugin_rpc: PluginRpc<'gpu>,
}
impl<'gpu> VgpuInstance<'gpu> {
+ /// Request the idempotent firmware release of this instance's CeUtils.
+ fn release_ceutils(
+ &self,
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ ) -> Result {
+ self.ceutils.release(dev, cmdq, bar)
+ }
+
/// Unmap the plugin communication buffer and return the slot release token.
fn unmap_and_take_slot(
self,
@@ -136,6 +151,47 @@ fn unmap_and_take_slot(
plugin_rpc.destroy(bar_user, mm)?;
Ok(vram_slot)
}
+
+ /// Scrub the instance framebuffer with its owned CeUtils allocation.
+ pub(crate) fn scrub_guest_fb(
+ &self,
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ ) -> Result {
+ self.ceutils
+ .scrub_guest_fb(dev, cmdq, bar, bar_user, mm, &self.vram_slot.fbmem)
+ }
+}
+
+/// Keep channel IDs unavailable when firmware ownership cannot be determined.
+fn quarantine_channel_ids(chids: ChannelIdReservation<'_>) {
+ // A failed GMC response cannot distinguish a command that was never
+ // executed from one whose reply was lost, and there is no ownership query
+ // for CeUtils. Running the reservation's destructor could therefore let a
+ // second owner reuse a firmware-owned CHID. Skipping it leaves those bits
+ // reserved for the remaining lifetime of the device's channel-ID pool.
+ core::mem::forget(chids);
+}
+
+/// Keep an invariant-violating slot and its backing VRAM unavailable for reuse.
+fn quarantine_vram_slot(slot: VgpuVramSlot) {
+ // A live slot without its allocator should be impossible. If it happens,
+ // stale BAR1 mappings may still refer to this VRAM. There is no recovery
+ // path without the allocator, so permanently retaining the backing
+ // allocation is safer than exposing it again.
+ core::mem::forget(slot);
+}
+
+/// Preserve every guard when publishing a fully built instance unexpectedly fails.
+fn quarantine_instance(instance: VgpuInstance<'_>) {
+ // allocate_instance() reserves registry capacity before acquiring any
+ // resource, so this is an invariant-failure fallback. If firmware release
+ // is also unconfirmed, retaining the complete instance prevents its CHID,
+ // BAR1 mapping, and VRAM from being independently reused.
+ core::mem::forget(instance);
}
/// Identity and firmware profile used to allocate an instance.
@@ -195,9 +251,7 @@ fn alloc_vram_slot(&mut self, mm: &GpuMm<'_>, layout: VgpuVramLayout) -> Result<
fn release_vram_slot(&mut self, slot: VgpuVramSlot) {
let Some(allocator) = self.vram_slots.as_mut() else {
- // A live slot proves that its pool exists. If that invariant is ever broken,
- // leaking the slot is safer than allowing its backing VRAM to be reused.
- core::mem::forget(slot);
+ quarantine_vram_slot(slot);
return;
};
allocator.release(slot);
@@ -205,9 +259,12 @@ fn release_vram_slot(&mut self, slot: VgpuVramSlot) {
/// Allocate resources, map the management communication region, and
/// register a new inactive vGPU instance.
+ #[expect(clippy::too_many_arguments)]
pub(crate) fn allocate_instance(
&mut self,
dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
bar_user: &BarUser<'gpu>,
mm: &mut GpuMm<'_>,
vgpu: &VgpuManager<'gpu>,
@@ -246,12 +303,14 @@ pub(crate) fn allocate_instance(
.total_channels()
.ok_or(ENODEV)?
.checked_div(vgpu_type.max_instance)
- .filter(|count| *count != 0)
+ .filter(|count| *count > 1)
.ok_or(EINVAL)?;
let chids = vgpu.chid_pool.reserve_ids(
NonZeroUsize::new(usize::try_from(num_chid).map_err(|_| EOVERFLOW)?).ok_or(EINVAL)?,
Alignment::SZ_1,
)?;
+ let ceutils_chid =
+ u32::try_from(chids.end.checked_sub(1).ok_or(EINVAL)?).map_err(|_| EOVERFLOW)?;
let layout = VgpuVramLayout {
type_id: vgpu_type.vgpu_type_id,
max_slots: vgpu_type.max_instance,
@@ -260,12 +319,58 @@ pub(crate) fn allocate_instance(
fb_align: vgpu.vmmu_segment_size().ok_or(ENODEV)?,
};
let vram_slot = self.alloc_vram_slot(mm, layout)?;
+ let ceutils = match CeUtils::allocate(dev, cmdq, bar, gfid, ceutils_chid, 0) {
+ Ok(ceutils) => ceutils,
+ Err(alloc_error) => {
+ let error = match alloc_error {
+ CeUtilsAllocError::NotOwned(error) => error,
+ CeUtilsAllocError::MayOwn(error) => {
+ if let Err(release_error) = CeUtils::release_gfid(dev, cmdq, bar, gfid) {
+ dev_err!(
+ dev,
+ "CeUtils alloc {:?}; firmware release unconfirmed: {:?}\n",
+ error,
+ release_error,
+ );
+ quarantine_channel_ids(chids);
+ }
+ error
+ }
+ };
+
+ // CeUtils allocation never receives the FB address, so the slot is safe to
+ // recycle once its local regions have been dropped.
+ self.release_vram_slot(vram_slot);
+ return Err(error);
+ }
+ };
+ if let Err(error) = ceutils.scrub_guest_fb(dev, cmdq, bar, bar_user, mm, &vram_slot.fbmem) {
+ // This error may be an unmap failure, or firmware may still be scrubbing. Keep the
+ // channel reservation and slot out of their allocators in either case.
+ quarantine_channel_ids(chids);
+ dev_err!(
+ dev,
+ "retaining CeUtils and VRAM slot {} after scrub error {:?}\n",
+ vram_slot.index(),
+ error,
+ );
+ return Err(error);
+ }
let comm = match CommBufferRegion::new(bar_user, mm, &vram_slot.mgmt_heap) {
Ok(comm) => comm,
Err(error) => {
// A failed page-table update may have installed a partial mapping without
// returning a handle that can unmap it. Keep the slot reserved so its backing
// VRAM cannot be reused while stale BAR1 PTEs may still reference it.
+ if let Err(release_error) = ceutils.release(dev, cmdq, bar) {
+ dev_err!(
+ dev,
+ "BAR1 error {:?}; CeUtils release unconfirmed: {:?}\n",
+ error,
+ release_error,
+ );
+ quarantine_channel_ids(chids);
+ }
dev_err!(
dev,
"allocate_instance: retaining slot {} after BAR1 map error {:?}\n",
@@ -283,22 +388,32 @@ pub(crate) fn allocate_instance(
vm_pid,
chids,
num_plugin_channels: 3,
+ ceutils,
vram_slot,
plugin_rpc: PluginRpc::new(comm),
};
match self.instances.push_within_capacity(instance) {
Ok(()) => Ok(gfid),
- Err(error) => match error.0.unmap_and_take_slot(bar_user, mm) {
- Ok(vram_slot) => {
- self.release_vram_slot(vram_slot);
- Err(EIO)
+ Err(error) => {
+ let instance = error.0;
+ if let Err(error) = instance.release_ceutils(dev, cmdq, bar) {
+ // Firmware may still own the final CHID. Keep every host resource
+ // quarantined rather than returning any of them to an allocator.
+ quarantine_instance(instance);
+ return Err(error);
}
- Err(error) => Err(error),
- },
+ match instance.unmap_and_take_slot(bar_user, mm) {
+ Ok(vram_slot) => {
+ self.release_vram_slot(vram_slot);
+ Err(EIO)
+ }
+ Err(error) => Err(error),
+ }
+ }
}
}
- /// Shut down and remove an instance, then release its reservations.
+ /// Shut down an instance, scrub its guest FB, and release its reservations.
pub(crate) fn destroy_instance(
&mut self,
dev: &device::Device<device::Bound>,
@@ -315,6 +430,8 @@ pub(crate) fn destroy_instance(
.ok_or(ENOENT)?;
shutdown(dev, cmdq, bar, gfid)?;
+ self.instances[index].scrub_guest_fb(dev, cmdq, bar, bar_user, mm)?;
+ self.instances[index].release_ceutils(dev, cmdq, bar)?;
cleanup(dev, cmdq, bar, gfid)?;
let instance = self.instances.remove(index).map_err(|_| EIO)?;
let vram_slot = instance.unmap_and_take_slot(bar_user, mm)?;
@@ -376,7 +493,7 @@ pub(crate) fn activate_instance(
instance.dbdf,
instance.vgpu_type.vgpu_type_id,
instance.vm_pid,
- u32::try_from(instance.chids.len()).map_err(|_| EOVERFLOW)?,
+ u32::try_from(instance.chids.len().checked_sub(1).ok_or(EINVAL)?).map_err(|_| EOVERFLOW)?,
instance.num_plugin_channels,
);
let gfid = instance.gfid;
diff --git a/drivers/gpu/nova-core/vgpu/mod.rs b/drivers/gpu/nova-core/vgpu/mod.rs
index 1c67d7afbe56..17e6d8d37af7 100644
--- a/drivers/gpu/nova-core/vgpu/mod.rs
+++ b/drivers/gpu/nova-core/vgpu/mod.rs
@@ -7,6 +7,7 @@
pub(crate) mod consts;
pub(crate) mod instance;
pub(crate) mod plugin_rpc;
+pub(crate) mod scrubber;
pub(crate) use self::instance::VgpuInstances;
diff --git a/drivers/gpu/nova-core/vgpu/scrubber.rs b/drivers/gpu/nova-core/vgpu/scrubber.rs
new file mode 100644
index 000000000000..0a5d60b7ea4a
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/scrubber.rs
@@ -0,0 +1,474 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+//! Per-VM CeUtils guest framebuffer scrubbing.
+
+use kernel::{
+ device,
+ prelude::*,
+ time::{
+ delay::fsleep,
+ Delta,
+ Instant,
+ Monotonic, //
+ },
+};
+
+use crate::{
+ driver::Bar0,
+ gsp::{
+ cmdq::Cmdq,
+ vgpu_bindings as bindings, //
+ },
+ mm::{
+ bar_user::{
+ Bar1Map,
+ BarUser, //
+ },
+ vram::VramRegion,
+ GpuMm,
+ Pfn,
+ VramAddress,
+ PAGE_SIZE, //
+ },
+ num,
+ vgpu::{
+ consts::gmc,
+ instance::Gfid, //
+ },
+};
+
+// OpenRM `channel_utils.h` defines `NV_CEUTILS_SEMA_PAGE_MAGIC` and places
+// `NV_CEUTILS_SEMA_PAGE_PAYLOAD_OFFSET` immediately after it.
+const NV_CEUTILS_SEMA_PAGE_MAGIC: u32 = 0xce5e_5ea0;
+
+#[repr(C)]
+struct CeUtilsSemaphoreHeader {
+ magic: u32,
+ payload: u32,
+}
+
+static_assert!(size_of::<CeUtilsSemaphoreHeader>() == 8);
+
+const SEMA_PAGE_MAGIC_OFFSET: usize = core::mem::offset_of!(CeUtilsSemaphoreHeader, magic);
+const SEMA_PAGE_PAYLOAD_OFFSET: usize = core::mem::offset_of!(CeUtilsSemaphoreHeader, payload);
+
+const SCRUB_REQUEST_SIZE: u64 = 4 * 1024 * 1024 * 1024;
+
+/// OpenRM uses a platform-dependent GPU timeout. Nova instead applies a fixed
+/// five-second host policy so that teardown cannot block a VFIO close forever;
+/// this is not a firmware ABI value.
+const SCRUB_TIMEOUT: Delta = Delta::from_secs(5);
+
+const MAGIC_HEAD: u32 = 0xdead_beef;
+const MAGIC_TAIL: u32 = 0xcafe_babe;
+
+#[repr(C)]
+#[derive(IntoBytes, zerocopy_derive::Immutable)]
+struct AllocCeutilsRequest {
+ gfid: u32,
+ fixed_chid: u32,
+ force_ceid: u32,
+ swizz_id: u32,
+}
+
+static_assert!(size_of::<AllocCeutilsRequest>() == 16);
+
+#[repr(C)]
+#[derive(FromBytes)]
+struct AllocCeutilsResponse {
+ semaphore_address: u64,
+ semaphore_aperture: u32,
+ _reserved: u32,
+}
+
+static_assert!(size_of::<AllocCeutilsResponse>() == 16);
+
+#[repr(C)]
+#[derive(IntoBytes, zerocopy_derive::Immutable)]
+struct FreeCeutilsRequest {
+ gfid: u32,
+}
+
+static_assert!(size_of::<FreeCeutilsRequest>() == 4);
+
+#[repr(C)]
+#[derive(IntoBytes, zerocopy_derive::Immutable)]
+struct ScrubGuestFbRequest {
+ gfid: u32,
+ reserved: u32,
+ fb_offset: u64,
+ fb_size: u64,
+}
+
+static_assert!(size_of::<ScrubGuestFbRequest>() == 24);
+
+#[repr(C)]
+#[derive(FromBytes)]
+struct ScrubGuestFbResponse {
+ work_id: u64,
+}
+
+static_assert!(size_of::<ScrubGuestFbResponse>() == 8);
+
+/// A firmware-owned per-VM CeUtils allocation.
+///
+/// The owner must call [`Self::release`] before returning its CHID or VRAM to
+/// their allocators.
+pub(crate) struct CeUtils {
+ gfid: Gfid,
+ chid: u32,
+ semaphore_address: u64,
+}
+
+/// Whether a failed allocation may still have transferred CHID ownership to firmware.
+pub(crate) enum CeUtilsAllocError {
+ /// A matching firmware response explicitly rejected the allocation.
+ NotOwned(Error),
+ /// The request may have completed despite a transport or response-validation error.
+ MayOwn(Error),
+}
+
+impl CeUtils {
+ /// Allocate a CeUtils channel and validate its semaphore description.
+ pub(crate) fn allocate(
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ gfid: Gfid,
+ chid: u32,
+ swizz_id: u32,
+ ) -> core::result::Result<Self, CeUtilsAllocError> {
+ let request = AllocCeutilsRequest {
+ gfid: gfid.0.to_le(),
+ fixed_chid: chid.to_le(),
+ force_ceid: u32::MAX.to_le(),
+ swizz_id: swizz_id.to_le(),
+ };
+
+ dev_dbg!(
+ dev,
+ "alloc CeUtils: gfid={} chid={} swizz_id={}\n",
+ gfid.0,
+ chid,
+ swizz_id,
+ );
+
+ let response = cmdq
+ .send_gmc_and_receive(
+ bar,
+ gmc::ALLOC_GSP_CEUTILS,
+ <AllocCeutilsRequest as IntoBytes>::as_bytes(&request),
+ num::usize_into_u32::<{ size_of::<AllocCeutilsResponse>() }>(),
+ )
+ .map_err(CeUtilsAllocError::MayOwn)?;
+ if response.status != 0 {
+ return Err(CeUtilsAllocError::NotOwned(EIO));
+ }
+
+ (|| {
+ let bytes = response
+ .payload
+ .get(..size_of::<AllocCeutilsResponse>())
+ .ok_or(EMSGSIZE)?;
+ let response = AllocCeutilsResponse::read_from_bytes(bytes).map_err(|_| EINVAL)?;
+ let semaphore_address = u64::from_le(response.semaphore_address);
+ let semaphore_aperture = u32::from_le(response.semaphore_aperture);
+ let page_size = u64::try_from(PAGE_SIZE).map_err(|_| EOVERFLOW)?;
+
+ if semaphore_address == 0
+ || !semaphore_address.is_multiple_of(page_size)
+ || semaphore_aperture != bindings::NV_ADDR_FBMEM
+ {
+ return Err(EINVAL);
+ }
+
+ dev_dbg!(
+ dev,
+ "alloc CeUtils: gfid={} semaphore={:#x}\n",
+ gfid.0,
+ semaphore_address,
+ );
+ Ok(Self {
+ gfid,
+ chid,
+ semaphore_address,
+ })
+ })()
+ .map_err(CeUtilsAllocError::MayOwn)
+ }
+
+ /// Scrub the complete guest framebuffer and verify its boundary markers.
+ pub(crate) fn scrub_guest_fb<'gpu>(
+ &self,
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ fb: &VramRegion,
+ ) -> Result {
+ write_markers(bar_user, mm, dev, fb)?;
+
+ let mut offset = fb.address();
+ let end = offset.checked_add(fb.size()).ok_or(EOVERFLOW)?;
+ while offset < end {
+ let size = core::cmp::min(SCRUB_REQUEST_SIZE, end - offset);
+ let work_id = submit_scrub(dev, cmdq, bar, self.gfid, offset, size)?;
+ wait_scrub_complete(bar_user, mm, dev, self.semaphore_address, work_id)?;
+ offset = offset.checked_add(size).ok_or(EOVERFLOW)?;
+ }
+
+ verify_markers_zeroed(bar_user, mm, dev, fb)
+ }
+
+ /// Request release of the firmware allocation.
+ ///
+ /// Firmware treats this operation as idempotent, but the GMC transaction
+ /// can fail after firmware has acted. An error therefore means that release
+ /// was not confirmed, and the caller must not return the CHID for reuse.
+ pub(crate) fn release(
+ &self,
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ ) -> Result {
+ dev_dbg!(
+ dev,
+ "free CeUtils: gfid={} chid={}\n",
+ self.gfid.0,
+ self.chid,
+ );
+ Self::release_gfid(dev, cmdq, bar, self.gfid)
+ }
+
+ /// Attempt an idempotent release when allocation ownership is uncertain.
+ pub(crate) fn release_gfid(
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ gfid: Gfid,
+ ) -> Result {
+ let request = FreeCeutilsRequest {
+ gfid: gfid.0.to_le(),
+ };
+
+ dev_dbg!(dev, "free CeUtils: gfid={}\n", gfid.0);
+ cmdq.send_gmc_and_check_status(
+ bar,
+ gmc::FREE_GSP_CEUTILS,
+ <FreeCeutilsRequest as IntoBytes>::as_bytes(&request),
+ )
+ }
+}
+
+/// Submit an asynchronous guest FB scrub and return its work identifier.
+fn submit_scrub(
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ gfid: Gfid,
+ fb_offset: u64,
+ fb_size: u64,
+) -> Result<u32> {
+ let request = ScrubGuestFbRequest {
+ gfid: gfid.0.to_le(),
+ reserved: 0,
+ fb_offset: fb_offset.to_le(),
+ fb_size: fb_size.to_le(),
+ };
+
+ dev_dbg!(
+ dev,
+ "submit scrub: gfid={} offset={:#x} size={:#x}\n",
+ gfid.0,
+ fb_offset,
+ fb_size,
+ );
+
+ let response = cmdq.send_gmc_and_receive(
+ bar,
+ gmc::SCRUB_GUEST_FB,
+ <ScrubGuestFbRequest as IntoBytes>::as_bytes(&request),
+ num::usize_into_u32::<{ size_of::<ScrubGuestFbResponse>() }>(),
+ )?;
+ if response.status != 0 {
+ return Err(EIO);
+ }
+
+ let bytes = response
+ .payload
+ .get(..size_of::<ScrubGuestFbResponse>())
+ .ok_or(EMSGSIZE)?;
+ let response = ScrubGuestFbResponse::read_from_bytes(bytes).map_err(|_| EINVAL)?;
+ let work_id = u32::try_from(u64::from_le(response.work_id)).map_err(|_| EOVERFLOW)?;
+ if work_id == 0 {
+ return Err(EIO);
+ }
+
+ Ok(work_id)
+}
+
+/// Poll the GSP-owned CeUtils semaphore page through a temporary BAR1 map.
+fn wait_scrub_complete<'gpu>(
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ dev: &device::Device<device::Bound>,
+ semaphore_address: u64,
+ work_id: u32,
+) -> Result {
+ let pfn = Pfn::from(VramAddress::from_raw(semaphore_address));
+ let semaphore_map = bar_user.map(mm, &[pfn], false)?;
+
+ let result = (|| {
+ let magic = semaphore_map.try_read32(SEMA_PAGE_MAGIC_OFFSET)?;
+ if magic != NV_CEUTILS_SEMA_PAGE_MAGIC {
+ dev_warn!(
+ dev,
+ "bad CeUtils semaphore magic {:#x}, expected {:#x}\n",
+ magic,
+ NV_CEUTILS_SEMA_PAGE_MAGIC,
+ );
+ return Err(EIO);
+ }
+
+ let start = Instant::<Monotonic>::now();
+ loop {
+ let value = semaphore_map.try_read32(SEMA_PAGE_PAYLOAD_OFFSET)?;
+ if value.wrapping_sub(work_id) < 0x8000_0000 {
+ dev_dbg!(
+ dev,
+ "scrub completed after {:?}: semaphore={:#x}, target={:#x}\n",
+ start.elapsed(),
+ value,
+ work_id,
+ );
+ return Ok(());
+ }
+
+ if start.elapsed() >= SCRUB_TIMEOUT {
+ dev_warn!(
+ dev,
+ "scrub timed out: semaphore={:#x}, target={:#x}\n",
+ value,
+ work_id,
+ );
+ return Err(ETIMEDOUT);
+ }
+ fsleep(Delta::from_millis(1));
+ }
+ })();
+
+ let cleanup = semaphore_map.release(mm);
+ match result {
+ Ok(()) => cleanup,
+ Err(error) => {
+ if let Err(cleanup_error) = cleanup {
+ dev_err!(
+ dev,
+ "failed to release semaphore BAR1 mapping after error {:?}: {:?}\n",
+ error,
+ cleanup_error,
+ );
+ }
+ Err(error)
+ }
+ }
+}
+
+fn with_bar1_map<'gpu, T>(
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ dev: &device::Device<device::Bound>,
+ region: VramRegion,
+ writable: bool,
+ operation: impl FnOnce(&Bar1Map<'gpu>) -> Result<T>,
+) -> Result<T> {
+ let map = Bar1Map::new(bar_user, mm, region, writable)?;
+ let result = operation(&map);
+ let cleanup = map.destroy(bar_user, mm);
+
+ match result {
+ Ok(value) => {
+ cleanup?;
+ Ok(value)
+ }
+ Err(error) => {
+ if let Err(cleanup_error) = cleanup {
+ dev_err!(
+ dev,
+ "failed to release temporary BAR1 mapping after error {:?}: {:?}\n",
+ error,
+ cleanup_error,
+ );
+ }
+ Err(error)
+ }
+ }
+}
+
+fn marker_regions(fb: &VramRegion) -> Result<(VramRegion, VramRegion, usize)> {
+ let page_size = u64::try_from(PAGE_SIZE).map_err(|_| EOVERFLOW)?;
+ let tail_page = fb.size().checked_sub(page_size).ok_or(EINVAL)?;
+ let tail_offset = PAGE_SIZE.checked_sub(size_of::<u32>()).ok_or(EOVERFLOW)?;
+
+ Ok((
+ fb.subregion(0..page_size)?,
+ fb.subregion(tail_page..fb.size())?,
+ tail_offset,
+ ))
+}
+
+/// Write and read back markers at the first and last framebuffer dwords.
+fn write_markers<'gpu>(
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ dev: &device::Device<device::Bound>,
+ fb: &VramRegion,
+) -> Result {
+ let (head_region, tail_region, tail_offset) = marker_regions(fb)?;
+
+ with_bar1_map(bar_user, mm, dev, head_region, true, |map| {
+ map.try_write32(MAGIC_HEAD, 0)?;
+ if map.try_read32(0)? != MAGIC_HEAD {
+ return Err(EIO);
+ }
+ Ok(())
+ })?;
+
+ with_bar1_map(bar_user, mm, dev, tail_region, true, |map| {
+ map.try_write32(MAGIC_TAIL, tail_offset)?;
+ if map.try_read32(tail_offset)? != MAGIC_TAIL {
+ return Err(EIO);
+ }
+ Ok(())
+ })
+}
+
+/// Verify that the first and last framebuffer dwords were zeroed.
+fn verify_markers_zeroed<'gpu>(
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ dev: &device::Device<device::Bound>,
+ fb: &VramRegion,
+) -> Result {
+ let (head_region, tail_region, tail_offset) = marker_regions(fb)?;
+ let head = with_bar1_map(bar_user, mm, dev, head_region, false, |map| {
+ map.try_read32(0)
+ })?;
+ let tail = with_bar1_map(bar_user, mm, dev, tail_region, false, |map| {
+ map.try_read32(tail_offset)
+ })?;
+
+ dev_dbg!(
+ dev,
+ "scrub markers: head={:#010x}, tail={:#010x}\n",
+ head,
+ tail,
+ );
+ if head != 0 || tail != 0 {
+ return Err(EIO);
+ }
+
+ Ok(())
+}
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 10/13] gpu: nova-core: vgpu: export plugin log buffers via debugfs
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
` (8 preceding siblings ...)
2026-09-05 8:11 ` [PATCH 09/13] gpu: nova-core: vgpu: scrub guest framebuffer memory with CeUtils Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 11/13] gpu: nova-core: vgpu: export lifecycle operations to VFIO Zhi Wang
` (2 subsequent siblings)
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
Expose the three per-VM GSP plugin log buffers (init, vgpu, kernel)
through debugfs so that nvlog_decoder can decode them at runtime.
Each log file is backed by VRAM in the management heap and read via
BAR1 MMIO. A self-describing header (architecture + build ID) is
prepended so decoding tools need no out-of-band metadata.
Make gpu.spec and gpu.build_id accessible to instance creation, and
promote LOG_BUFFER_HEADER_SIZE / build_log_buffer_header to pub(crate)
for reuse by the vGPU log module.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/driver.rs | 42 ++++--
drivers/gpu/nova-core/gpu.rs | 10 +-
drivers/gpu/nova-core/gsp.rs | 16 ++-
drivers/gpu/nova-core/mm/bar_user.rs | 61 ++++++---
drivers/gpu/nova-core/vgpu/fw.rs | 117 +++++++++++++++-
drivers/gpu/nova-core/vgpu/instance.rs | 79 ++++++++++-
drivers/gpu/nova-core/vgpu/log.rs | 167 +++++++++++++++++++++++
drivers/gpu/nova-core/vgpu/mod.rs | 1 +
drivers/gpu/nova-core/vgpu/plugin_rpc.rs | 6 +
9 files changed, 458 insertions(+), 41 deletions(-)
create mode 100644 drivers/gpu/nova-core/vgpu/log.rs
diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver.rs
index c4b9ac03d3a1..4648f4a2e089 100644
--- a/drivers/gpu/nova-core/driver.rs
+++ b/drivers/gpu/nova-core/driver.rs
@@ -6,6 +6,7 @@
Bound,
Core, //
},
+ devres::Devres,
io::resource,
pci,
pci::{
@@ -15,9 +16,12 @@
},
prelude::*,
sizes::SZ_16M,
- sync::atomic::{
- Atomic,
- Relaxed, //
+ sync::{
+ atomic::{
+ Atomic,
+ Relaxed, //
+ },
+ Arc, //
},
types::ForLt,
};
@@ -35,18 +39,28 @@
#[pin_data]
pub(crate) struct NovaCore<'bound> {
+ /// Auxiliary DRM-device registration.
+ ///
+ /// Declared first so consumers are unregistered before the interrupt and GPU resources are
+ /// torn down.
+ #[allow(clippy::type_complexity)]
+ _reg: auxiliary::Registration<'bound, ForLt!(())>,
/// GSP event interrupt registration.
///
- /// Declared first so it is dropped first: `free_irq` runs (waiting out any in-flight handler)
- /// before the GSP is unloaded (`gpu`) or the BAR mapping is released (`bar`).
+ /// Declared before the GPU and BAR resources so `free_irq` runs (waiting out any in-flight
+ /// handler) before the GSP is unloaded (`gpu`) or the BAR mapping is released (`bar`).
#[pin]
_gsp_irq: GspIrq<'bound>,
#[pin]
pub(crate) gpu: Gpu<'bound>,
bar: pci::Bar<'bound, BAR0_SIZE>,
- bar1: Bar1<'bound>,
- #[allow(clippy::type_complexity)]
- _reg: auxiliary::Registration<'bound, ForLt!(())>,
+ /// Device-managed BAR1 mapping shared with debugfs readers.
+ ///
+ /// Debugfs file backing types must be `'static`, so readers cannot retain
+ /// the lifetime-bound [`Bar1`] reference used before log export was added.
+ /// [`Devres`] revokes access during unbind, while [`Arc`] keeps the handle
+ /// alive until all scoped readers have drained.
+ bar1: Arc<Devres<Bar1<'static>>>,
/// Self-referential borrow of `vectors`, so this does not have to be repeated in the
/// constructor. Will go away with self-referential pin-init.
vectors_ref: &'bound SubtreeVectors<'bound>,
@@ -129,17 +143,17 @@ fn probe<'bound>(
// is dropped after all fields that use `vectors_ref` (struct field drop order).
vectors_ref: unsafe { &*core::ptr::from_ref(vectors.as_ref().get_ref()) },
bar: pdev.iomap_region_sized::<BAR0_SIZE>(0, c"nova-core/bar0")?,
- bar1: {
- let bar1_idx = bar1_resource_index(pdev)?;
- pdev.iomap_region(bar1_idx, c"nova-core/bar1")?
- },
+ bar1: Arc::new(
+ pdev.iomap_region(bar1_resource_index(pdev)?, c"nova-core/bar1")?
+ .into_devres()?,
+ GFP_KERNEL,
+ )?,
// TODO: Use self-referential pin-init syntax once available.
gpu <- Gpu::new(
pdev,
// SAFETY: `bar` is initialized above, pinned, and outlives `gpu`.
unsafe { &*core::ptr::from_ref(bar) },
- // SAFETY: `bar1` is initialized above, pinned, and outlives `gpu`.
- unsafe { &*core::ptr::from_ref(bar1) },
+ bar1.clone(),
vectors_ref,
),
// Quiesce the interrupt tree before registering the handler below.
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index 5c12847c19bc..f88be4ca6ea5 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -4,6 +4,7 @@
use kernel::{
device,
+ devres::Devres,
dma::Device,
fmt,
gpu::buddy::GpuBuddyParams,
@@ -31,6 +32,7 @@
Falcon, //
},
fb::SysmemFlush,
+ firmware,
fsp::Fsp,
gsp::{
self,
@@ -374,10 +376,16 @@ pub(crate) fn cmdq(&self) -> Arc<Cmdq> {
self.gsp_resources.gsp.cmdq()
}
+ /// Returns the firmware build identifier, if one was reported.
+ #[expect(dead_code)]
+ pub(crate) fn build_id(&self) -> Option<firmware::BuildId> {
+ self.gsp_resources.gsp.build_id()
+ }
+
pub(crate) fn new(
pdev: &'gpu pci::Device<device::Core<'_>>,
bar: Bar0<'gpu>,
- bar1: &'gpu Bar1<'gpu>,
+ bar1: Arc<Devres<Bar1<'static>>>,
vectors: &'gpu SubtreeVectors<'gpu>,
) -> impl PinInit<Self, Error> + 'gpu {
let dev = pdev.as_ref();
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index 2521d7331996..210bf8de0633 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -116,7 +116,10 @@ fn init(view: CoherentView<'_, Self>, start: DmaAddress) -> Result<()> {
/// This header makes each dump self-describing so that decoding tools can
/// identify the firmware build, GPU architecture, and metadata format without
/// out-of-band information.
-const LOG_BUFFER_HEADER_SIZE: usize = 0x48;
+///
+/// `0x48` is the offset of `data` in `LIBOS_LOG_NVLOG_BUFFER_V2`, as defined
+/// by Open RM's `uproc/os/common/include/liblogdecode.h`.
+pub(crate) const LOG_BUFFER_HEADER_SIZE: usize = 0x48;
/// Build a log buffer header from GPU and firmware metadata.
///
@@ -130,7 +133,7 @@ fn init(view: CoherentView<'_, Self>, start: DmaAddress) -> Result<()> {
/// 0x20 buildId[32]
/// 0x40 flags (u32) = 1 (packed metadata)
/// 0x44 reserved (u32) = 0
-fn build_log_buffer_header(
+pub(crate) fn build_log_buffer_header(
chipset: Chipset,
build_id: &BuildId,
task_prefix: &str,
@@ -411,6 +414,15 @@ pub(crate) const fn vgpu_state(&self) -> VgpuState {
pub(crate) fn cmdq(&self) -> Arc<Cmdq> {
self.cmdq.clone()
}
+
+ /// Returns the firmware build identifier, if one was reported.
+ pub(crate) fn build_id(&self) -> Option<BuildId> {
+ Tlv::new(self.gsp_tlv.data())
+ .ok()?
+ .get_bytes(b"BLID")
+ .ok()
+ .and_then(BuildId::from_raw)
+ }
}
/// Opaque bundle required to unload the GSP. Created by [`Gsp::boot`], consumed by [`Gsp::unload`].
diff --git a/drivers/gpu/nova-core/mm/bar_user.rs b/drivers/gpu/nova-core/mm/bar_user.rs
index 158160fbf352..e7db8c5f1232 100644
--- a/drivers/gpu/nova-core/mm/bar_user.rs
+++ b/drivers/gpu/nova-core/mm/bar_user.rs
@@ -3,11 +3,17 @@
//! BAR1 user interface for CPU access to GPU virtual memory. Used for USERD
//! for GPU work submission, and applications to access GPU buffers via mmap().
+use core::marker::PhantomData;
+
use kernel::{
+ devres::Devres,
io::Io,
new_mutex,
prelude::*,
- sync::Mutex, //
+ sync::{
+ Arc,
+ Mutex, //
+ },
};
use crate::{
@@ -39,7 +45,8 @@
pub(crate) struct BarUser<'gpu> {
#[pin]
vmm: Mutex<Vmm>,
- bar1: &'gpu Bar1<'gpu>,
+ bar1: Arc<Devres<Bar1<'static>>>,
+ _gpu: PhantomData<&'gpu ()>,
}
impl<'gpu> BarUser<'gpu> {
@@ -48,12 +55,13 @@ pub(crate) fn new(
pdb_addr: VramAddress,
chipset: Chipset,
va_size: u64,
- bar1: &'gpu Bar1<'gpu>,
+ bar1: Arc<Devres<Bar1<'static>>>,
) -> Result<impl PinInit<Self> + 'gpu> {
let vmm = Vmm::new(pdb_addr, chipset.mmu_version(), va_size)?;
Ok(pin_init!(Self {
vmm <- new_mutex!(vmm, "bar_user_vmm"),
bar1,
+ _gpu: PhantomData,
}))
}
@@ -138,31 +146,36 @@ fn bar_offset(&self, offset: usize) -> Result<usize> {
/// Read a 32-bit value at the given offset.
pub(crate) fn try_read32(&self, offset: usize) -> Result<u32> {
let off = self.bar_offset(offset)?;
- self.bar_user.bar1.try_read32(off)
+ let bar1 = self.bar_user.bar1.try_access().ok_or(ENXIO)?;
+ bar1.try_read32(off)
}
/// Write an 8-bit value at the given offset.
pub(crate) fn try_write8(&self, value: u8, offset: usize) -> Result {
let off = self.bar_offset(offset)?;
- self.bar_user.bar1.try_write8(value, off)
+ let bar1 = self.bar_user.bar1.try_access().ok_or(ENXIO)?;
+ bar1.try_write8(value, off)
}
/// Write a 32-bit value at the given offset.
pub(crate) fn try_write32(&self, value: u32, offset: usize) -> Result {
let off = self.bar_offset(offset)?;
- self.bar_user.bar1.try_write32(value, off)
+ let bar1 = self.bar_user.bar1.try_access().ok_or(ENXIO)?;
+ bar1.try_write32(value, off)
}
/// Read a 64-bit value at the given offset.
pub(crate) fn try_read64(&self, offset: usize) -> Result<u64> {
let off = self.bar_offset(offset)?;
- self.bar_user.bar1.try_read64(off)
+ let bar1 = self.bar_user.bar1.try_access().ok_or(ENXIO)?;
+ bar1.try_read64(off)
}
/// Write a 64-bit value at the given offset.
pub(crate) fn try_write64(&self, value: u64, offset: usize) -> Result {
let off = self.bar_offset(offset)?;
- self.bar_user.bar1.try_write64(value, off)
+ let bar1 = self.bar_user.bar1.try_access().ok_or(ENXIO)?;
+ bar1.try_write64(value, off)
}
}
@@ -184,11 +197,12 @@ fn drop(&mut self) {
/// logical region may begin or end within a page; the containing pages are mapped while CPU
/// access remains bounded to the requested byte range.
pub(crate) struct Bar1Map<'gpu> {
- bar1: &'gpu Bar1<'gpu>,
+ bar1: Arc<Devres<Bar1<'static>>>,
mapped: MappedRange,
region: VramRegion,
page_bias: usize,
logical_size: usize,
+ _gpu: PhantomData<&'gpu ()>,
}
impl<'gpu> Bar1Map<'gpu> {
@@ -227,14 +241,20 @@ pub(crate) fn new(
let mapped = vmm.map_pages(mm, &pfns, None, writable)?;
Ok(Self {
- bar1: bar_user.bar1,
+ bar1: bar_user.bar1.clone(),
mapped,
region,
page_bias,
logical_size,
+ _gpu: PhantomData,
})
}
+ /// Clone the revocable BAR1 mapping used by debugfs readers.
+ pub(crate) fn bar1_arc(&self) -> &Arc<Devres<Bar1<'static>>> {
+ &self.bar1
+ }
+
/// Returns the mapped physical VRAM region.
pub(crate) fn region(&self) -> &VramRegion {
&self.region
@@ -272,23 +292,23 @@ fn bar_offset(&self, offset: usize, width: usize) -> Result<usize> {
// BAR1 and the logical mapping have runtime sizes, so these accessors
// validate the offset, width, and alignment before performing MMIO.
pub(crate) fn try_read32(&self, offset: usize) -> Result<u32> {
- self.bar1
- .try_read32(self.bar_offset(offset, size_of::<u32>())?)
+ let bar1 = self.bar1.try_access().ok_or(ENXIO)?;
+ bar1.try_read32(self.bar_offset(offset, size_of::<u32>())?)
}
pub(crate) fn try_write8(&self, value: u8, offset: usize) -> Result {
- self.bar1
- .try_write8(value, self.bar_offset(offset, size_of::<u8>())?)
+ let bar1 = self.bar1.try_access().ok_or(ENXIO)?;
+ bar1.try_write8(value, self.bar_offset(offset, size_of::<u8>())?)
}
pub(crate) fn try_write32(&self, value: u32, offset: usize) -> Result {
- self.bar1
- .try_write32(value, self.bar_offset(offset, size_of::<u32>())?)
+ let bar1 = self.bar1.try_access().ok_or(ENXIO)?;
+ bar1.try_write32(value, self.bar_offset(offset, size_of::<u32>())?)
}
pub(crate) fn try_write64(&self, value: u64, offset: usize) -> Result {
- self.bar1
- .try_write64(value, self.bar_offset(offset, size_of::<u64>())?)
+ let bar1 = self.bar1.try_access().ok_or(ENXIO)?;
+ bar1.try_write64(value, self.bar_offset(offset, size_of::<u64>())?)
}
/// Invalidates the PTEs and releases the BAR1 virtual address.
@@ -331,7 +351,10 @@ pub(crate) fn run_self_test(
const PATTERN_PRAMIN: u32 = 0xDEAD_BEEF;
const PATTERN_BAR1: u32 = 0xCAFE_BABE;
- let bar1 = bar_user.bar1;
+ // A matching bound device proves that devres cannot be revoked while this
+ // self-test runs, so this reference may safely span allocations and other
+ // potentially sleeping operations below.
+ let bar1 = bar_user.bar1.access(dev)?;
dev_info!(dev, "MM: Starting self-test...\n");
let pdb_addr = VramAddress::from_raw(bar1_pdb);
diff --git a/drivers/gpu/nova-core/vgpu/fw.rs b/drivers/gpu/nova-core/vgpu/fw.rs
index db6f535998a9..af7e86405b9e 100644
--- a/drivers/gpu/nova-core/vgpu/fw.rs
+++ b/drivers/gpu/nova-core/vgpu/fw.rs
@@ -8,9 +8,15 @@
RpcResponse, //
};
-use kernel::prelude::*;
+use kernel::{
+ devres::Devres,
+ io::Io,
+ prelude::*,
+ sync::Arc, //
+};
use crate::{
+ driver::Bar1,
gsp::vgpu_bindings as bindings,
mm::{
bar_user::{
@@ -57,6 +63,104 @@ fn take_region(region: &VramRegion, cursor: &mut u64, size: u32) -> Result<VramR
Ok(subregion)
}
+/// Revocable BAR1 view of one vGPU plugin log buffer.
+pub(crate) struct MappedPluginLogBuffer {
+ bar1: Arc<Devres<Bar1<'static>>>,
+ gpu_va_addr: usize,
+ size: usize,
+}
+
+impl MappedPluginLogBuffer {
+ fn new(map: &Bar1Map<'_>, region: &VramRegion) -> Result<Self> {
+ let start = region
+ .address()
+ .checked_sub(map.region().address())
+ .ok_or(EINVAL)
+ .and_then(|start| usize::try_from(start).map_err(|_| EOVERFLOW))?;
+ let size = usize::try_from(region.size()).map_err(|_| EOVERFLOW)?;
+ let end = start.checked_add(size).ok_or(EOVERFLOW)?;
+ if end > map.size() || !start.is_multiple_of(4) || !size.is_multiple_of(4) {
+ return Err(EINVAL);
+ }
+
+ let gpu_va_addr = usize::try_from(map.gpu_va_addr()?)
+ .map_err(|_| EOVERFLOW)?
+ .checked_add(start)
+ .ok_or(EOVERFLOW)?;
+ if !gpu_va_addr.is_multiple_of(4) {
+ return Err(EINVAL);
+ }
+
+ let bar1 = map.bar1_arc().clone();
+ {
+ let mapped_bar1 = bar1.try_access().ok_or(ENXIO)?;
+ if gpu_va_addr.checked_add(size).ok_or(EOVERFLOW)? > mapped_bar1.size() {
+ return Err(EINVAL);
+ }
+ }
+
+ Ok(Self {
+ bar1,
+ gpu_va_addr,
+ size,
+ })
+ }
+
+ /// Return the log buffer size in bytes.
+ pub(crate) const fn size(&self) -> usize {
+ self.size
+ }
+
+ /// Stage a range of log bytes in a kernel buffer.
+ pub(crate) fn read(&self, offset: usize, output: &mut [u8]) -> Result {
+ let end = offset.checked_add(output.len()).ok_or(EOVERFLOW)?;
+ if end > self.size {
+ return Err(EINVAL);
+ }
+
+ let bar1 = self.bar1.try_access().ok_or(ENXIO)?;
+ let mut source = offset;
+ let mut copied = 0usize;
+
+ while copied < output.len() {
+ let aligned_source = source & !3;
+ let within = source & 3;
+ let bar_offset = self
+ .gpu_va_addr
+ .checked_add(aligned_source)
+ .ok_or(EOVERFLOW)?;
+ let bytes = bar1.try_read32(bar_offset)?.to_le_bytes();
+ let chunk = (4 - within).min(output.len() - copied);
+
+ output[copied..copied + chunk].copy_from_slice(&bytes[within..within + chunk]);
+ source = source.checked_add(chunk).ok_or(EOVERFLOW)?;
+ copied += chunk;
+ }
+
+ Ok(())
+ }
+}
+
+/// Revocable BAR1 views of all vGPU plugin log buffers.
+pub(crate) struct MappedPluginLogBuffers {
+ init: MappedPluginLogBuffer,
+ vgpu: MappedPluginLogBuffer,
+ kernel: MappedPluginLogBuffer,
+}
+
+impl MappedPluginLogBuffers {
+ /// Split the aggregate into its three task log views.
+ pub(crate) fn into_parts(
+ self,
+ ) -> (
+ MappedPluginLogBuffer,
+ MappedPluginLogBuffer,
+ MappedPluginLogBuffer,
+ ) {
+ (self.init, self.vgpu, self.kernel)
+ }
+}
+
/// BAR1 mapping and semantic regions of a vGPU CPU-GSP communication buffer.
///
/// The host and GSP plugin exchange control, response, message, migration,
@@ -214,6 +318,17 @@ pub(crate) fn plugin_logs(&self) -> Result<PluginLogRegions> {
})
}
+ /// Return revocable BAR1 views of the three plugin logs.
+ pub(crate) fn mapped_plugin_logs(&self) -> Result<MappedPluginLogBuffers> {
+ let logs = self.plugin_logs()?;
+
+ Ok(MappedPluginLogBuffers {
+ init: MappedPluginLogBuffer::new(&self.map, logs.init())?,
+ vgpu: MappedPluginLogBuffer::new(&self.map, logs.vgpu())?,
+ kernel: MappedPluginLogBuffer::new(&self.map, logs.kernel())?,
+ })
+ }
+
/// Return whether firmware has published the plugin boot marker.
pub(crate) fn is_plugin_ready(&self) -> Result<bool> {
let value = self.read_u32(
diff --git a/drivers/gpu/nova-core/vgpu/instance.rs b/drivers/gpu/nova-core/vgpu/instance.rs
index d6938cc65bf9..ed715951e92d 100644
--- a/drivers/gpu/nova-core/vgpu/instance.rs
+++ b/drivers/gpu/nova-core/vgpu/instance.rs
@@ -4,15 +4,21 @@
use core::num::NonZeroUsize;
use kernel::{
+ debugfs,
device,
prelude::*,
ptr::Alignment,
- sizes::SizeConstants, //
+ sizes::SizeConstants,
+ str::CString, //
};
use crate::{
driver::Bar0,
- gpu::ChannelIdReservation,
+ firmware::BuildId,
+ gpu::{
+ ChannelIdReservation,
+ Chipset, //
+ },
gsp::{
cmdq::Cmdq,
commands::{
@@ -33,7 +39,11 @@
shutdown, //
},
consts::gmc,
- fw::CommBufferRegion,
+ fw::{
+ CommBufferRegion,
+ MappedPluginLogBuffers, //
+ },
+ log::VgpuLogBuffers,
plugin_rpc::{
PluginConfigParams,
PluginRpc, //
@@ -112,7 +122,11 @@ fn from_properties(properties: &VgpuProperties) -> Self {
}
}
-/// A vGPU instance and the resources reserved for it.
+/// A live vGPU instance with allocated resources.
+///
+/// Field ordering is load-bearing for drop: `debugfs_logs` must be declared
+/// before `plugin_rpc` so that debugfs entries are removed (and in-progress
+/// readers drained) before the underlying `Bar1Map` is destroyed.
#[expect(dead_code)]
pub(crate) struct VgpuInstance<'gpu> {
pub(crate) gfid: Gfid,
@@ -123,6 +137,7 @@ pub(crate) struct VgpuInstance<'gpu> {
pub(crate) num_plugin_channels: u32,
ceutils: CeUtils,
pub(crate) vram_slot: VgpuVramSlot,
+ debugfs_logs: Option<Pin<KBox<debugfs::Scope<VgpuLogBuffers>>>>,
pub(crate) plugin_rpc: PluginRpc<'gpu>,
}
@@ -144,10 +159,14 @@ fn unmap_and_take_slot(
mm: &mut GpuMm<'_>,
) -> Result<VgpuVramSlot> {
let Self {
+ debugfs_logs,
plugin_rpc,
vram_slot,
..
} = self;
+ // Remove the files and drain active readers before tearing down the
+ // BAR1 mapping that backs them.
+ drop(debugfs_logs);
plugin_rpc.destroy(bar_user, mm)?;
Ok(vram_slot)
}
@@ -214,6 +233,39 @@ pub(crate) const fn new(gfid: Gfid, dbdf: Dbdf, vgpu_type: VgpuType, vm_pid: u32
}
}
+fn create_debugfs_logs(
+ buffers: MappedPluginLogBuffers,
+ dbdf: Dbdf,
+ chipset: Chipset,
+ build_id: Option<&BuildId>,
+) -> Result<Pin<KBox<debugfs::Scope<VgpuLogBuffers>>>> {
+ let logs = VgpuLogBuffers::new(buffers, chipset, build_id)?;
+ let raw_dbdf = dbdf.into_raw();
+ let domain = raw_dbdf >> 16;
+ let bus = (raw_dbdf >> 8) & 0xff;
+ let device = (raw_dbdf >> 3) & 0x1f;
+ let function = raw_dbdf & 0x07;
+ let directory = CString::try_from_fmt(fmt!(
+ "{:04x}:{:02x}:{:02x}.{:x}-vgpu",
+ domain,
+ bus,
+ device,
+ function,
+ ))?;
+
+ #[allow(static_mut_refs)]
+ // SAFETY: The root is initialized before driver registration and cleared
+ // only after driver unregistration has drained all users.
+ let root = unsafe { crate::DEBUGFS_ROOT.as_ref() }.ok_or(ENODEV)?;
+
+ KBox::pin_init(
+ root.scope(logs, &directory, |logs, directory| {
+ VgpuLogBuffers::register_debugfs(logs, directory);
+ }),
+ GFP_KERNEL,
+ )
+}
+
/// Registry of live vGPU instances.
pub(crate) struct VgpuInstances<'gpu> {
/// Declared before `vram_slots` so instance regions are dropped before their backing pool.
@@ -390,6 +442,7 @@ pub(crate) fn allocate_instance(
num_plugin_channels: 3,
ceutils,
vram_slot,
+ debugfs_logs: None,
plugin_rpc: PluginRpc::new(comm),
};
match self.instances.push_within_capacity(instance) {
@@ -485,6 +538,8 @@ pub(crate) fn activate_instance(
bar: Bar0<'_>,
instance: &mut VgpuInstance<'_>,
fifo_engine_list: &FifoEngineList,
+ chipset: Chipset,
+ build_id: Option<&BuildId>,
) -> Result {
bootload(dev, cmdq, bar, instance, fifo_engine_list)?;
@@ -503,5 +558,21 @@ pub(crate) fn activate_instance(
rpc.send_config_params(dev, bar, gfid, ¶ms)?;
rpc.set_bme(dev, bar, gfid, true)?;
+ // Publish the log files only after the plugin has initialized its
+ // management heap. Debugfs is diagnostic, so failure must not undo an
+ // otherwise usable vGPU instance.
+ match rpc
+ .mapped_plugin_logs()
+ .and_then(|buffers| create_debugfs_logs(buffers, instance.dbdf, chipset, build_id))
+ {
+ Ok(logs) => instance.debugfs_logs = Some(logs),
+ Err(error) => dev_warn!(
+ dev,
+ "debugfs logs unavailable for gfid={}: {:?}\n",
+ gfid.0,
+ error,
+ ),
+ }
+
Ok(())
}
diff --git a/drivers/gpu/nova-core/vgpu/log.rs b/drivers/gpu/nova-core/vgpu/log.rs
new file mode 100644
index 000000000000..6da92b0fd789
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/log.rs
@@ -0,0 +1,167 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+use kernel::{
+ debugfs,
+ fs::file,
+ prelude::*,
+ uaccess::UserSliceWriter, //
+};
+
+use crate::{
+ firmware::BuildId,
+ gpu::Chipset,
+ gsp::{
+ build_log_buffer_header,
+ LOG_BUFFER_HEADER_SIZE, //
+ },
+ vgpu::fw::{
+ MappedPluginLogBuffer,
+ MappedPluginLogBuffers, //
+ },
+};
+
+const LOG_READ_CHUNK_SIZE: usize = 4096;
+
+/// A single vGPU plugin log buffer backed by VRAM, read via BAR1 MMIO.
+///
+/// The GSP plugin writes encoded log entries into the management heap in
+/// VRAM. The mapped buffer retains revocable access to those bytes without a
+/// device reference.
+///
+/// A [`LOG_BUFFER_HEADER_SIZE`]-byte header is prepended so that
+/// `nvlog_decoder` can identify the GPU architecture and firmware build.
+pub(crate) struct VgpuLogBuffer {
+ buffer: MappedPluginLogBuffer,
+ header: [u8; LOG_BUFFER_HEADER_SIZE],
+ header_len: usize,
+}
+
+impl VgpuLogBuffer {
+ fn new(
+ buffer: MappedPluginLogBuffer,
+ chipset: Chipset,
+ build_id: Option<&BuildId>,
+ task_prefix: &str,
+ ) -> Result<Self> {
+ let (header, header_len) = match build_id {
+ Some(bid) => (
+ build_log_buffer_header(chipset, bid, task_prefix),
+ LOG_BUFFER_HEADER_SIZE,
+ ),
+ None => ([0u8; LOG_BUFFER_HEADER_SIZE], 0),
+ };
+
+ Ok(Self {
+ buffer,
+ header,
+ header_len,
+ })
+ }
+}
+
+impl debugfs::BinaryWriter for VgpuLogBuffer {
+ fn write_to_slice(
+ &self,
+ writer: &mut UserSliceWriter,
+ offset: &mut file::Offset,
+ ) -> Result<usize> {
+ if offset.is_negative() {
+ return Err(EINVAL);
+ }
+
+ let offset_val: usize = (*offset).try_into().map_err(|_| EINVAL)?;
+ let total_len = self
+ .header_len
+ .checked_add(self.buffer.size())
+ .ok_or(EOVERFLOW)?;
+
+ if offset_val >= total_len {
+ return Ok(0);
+ }
+
+ let count = (total_len - offset_val).min(writer.len());
+ if count == 0 {
+ return Ok(0);
+ }
+
+ // Keep the staging buffer on the heap to avoid putting a page-sized
+ // object on the kernel stack.
+ let staging_size = count.min(LOG_READ_CHUNK_SIZE);
+ let mut staging = KVec::new();
+ staging.resize(staging_size, 0, GFP_KERNEL)?;
+
+ let mut written = 0usize;
+ while written < count {
+ let chunk_len = (count - written).min(staging.len());
+ let chunk = &mut staging[..chunk_len];
+ let chunk_offset = offset_val.checked_add(written).ok_or(EOVERFLOW)?;
+ let mut filled = 0usize;
+
+ if chunk_offset < self.header_len {
+ let header_len = (self.header_len - chunk_offset).min(chunk_len);
+ chunk[..header_len]
+ .copy_from_slice(&self.header[chunk_offset..chunk_offset + header_len]);
+ filled = header_len;
+ }
+
+ if filled < chunk_len {
+ let log_offset = chunk_offset
+ .checked_add(filled)
+ .ok_or(EOVERFLOW)?
+ .checked_sub(self.header_len)
+ .ok_or(EINVAL)?;
+
+ // The mapped buffer drops its revocable access guard before
+ // the potentially sleeping userspace copy below.
+ self.buffer.read(log_offset, &mut chunk[filled..])?;
+ }
+
+ writer.write_slice(chunk)?;
+ written = written.checked_add(chunk_len).ok_or(EOVERFLOW)?;
+ }
+
+ *offset = (*offset)
+ .checked_add(i64::try_from(written).map_err(|_| EOVERFLOW)?)
+ .ok_or(EOVERFLOW)?;
+ Ok(written)
+ }
+}
+
+/// Aggregated log buffers for a single vGPU instance.
+///
+/// Each vGPU plugin produces three log streams within the management heap:
+/// - `init_log`: init task log (128 KB)
+/// - `vgpu_log`: vGPU task log (256 KB)
+/// - `kernel_log`: kernel task log (64 KB)
+pub(crate) struct VgpuLogBuffers {
+ init_log: VgpuLogBuffer,
+ vgpu_log: VgpuLogBuffer,
+ kernel_log: VgpuLogBuffer,
+}
+
+impl VgpuLogBuffers {
+ pub(crate) fn new(
+ buffers: MappedPluginLogBuffers,
+ chipset: Chipset,
+ build_id: Option<&BuildId>,
+ ) -> Result<Self> {
+ let (init, vgpu, kernel) = buffers.into_parts();
+
+ Ok(Self {
+ init_log: VgpuLogBuffer::new(init, chipset, build_id, "INIT")?,
+ vgpu_log: VgpuLogBuffer::new(vgpu, chipset, build_id, "VGPU")?,
+ kernel_log: VgpuLogBuffer::new(kernel, chipset, build_id, "KERN")?,
+ })
+ }
+
+ /// Register debugfs binary files for these log buffers within a scoped directory.
+ pub(crate) fn register_debugfs<'data, 'dir>(
+ logs: &'data VgpuLogBuffers,
+ dir: &'dir debugfs::ScopedDir<'data, 'dir>,
+ ) {
+ dir.read_binary_file(c"init_log", &logs.init_log);
+ dir.read_binary_file(c"vgpu_log", &logs.vgpu_log);
+ dir.read_binary_file(c"kernel_log", &logs.kernel_log);
+ }
+}
diff --git a/drivers/gpu/nova-core/vgpu/mod.rs b/drivers/gpu/nova-core/vgpu/mod.rs
index 17e6d8d37af7..61f799c2748e 100644
--- a/drivers/gpu/nova-core/vgpu/mod.rs
+++ b/drivers/gpu/nova-core/vgpu/mod.rs
@@ -6,6 +6,7 @@
pub(crate) mod bootload;
pub(crate) mod consts;
pub(crate) mod instance;
+pub(crate) mod log;
pub(crate) mod plugin_rpc;
pub(crate) mod scrubber;
diff --git a/drivers/gpu/nova-core/vgpu/plugin_rpc.rs b/drivers/gpu/nova-core/vgpu/plugin_rpc.rs
index f1fa6fe238fb..4b759ec0bf42 100644
--- a/drivers/gpu/nova-core/vgpu/plugin_rpc.rs
+++ b/drivers/gpu/nova-core/vgpu/plugin_rpc.rs
@@ -27,6 +27,7 @@
},
vgpu::fw::{
CommBufferRegion,
+ MappedPluginLogBuffers,
PluginLogRegions,
RpcMessage,
RpcResponse, //
@@ -92,6 +93,11 @@ pub(crate) fn plugin_logs(&self) -> Result<PluginLogRegions> {
self.comm.plugin_logs()
}
+ /// Return revocable BAR1 views of the plugin logs.
+ pub(crate) fn mapped_plugin_logs(&self) -> Result<MappedPluginLogBuffers> {
+ self.comm.mapped_plugin_logs()
+ }
+
/// Poll the control buffer until the plugin publishes its boot marker.
pub(crate) fn wait_plugin_ready(&self, dev: &device::Device<device::Bound>) -> Result {
let start = Instant::<Monotonic>::now();
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 11/13] gpu: nova-core: vgpu: export lifecycle operations to VFIO
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
` (9 preceding siblings ...)
2026-09-05 8:11 ` [PATCH 10/13] gpu: nova-core: vgpu: export plugin log buffers via debugfs Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-05 8:11 ` [PATCH 12/13] vfio/nvidia-vgpu: add the NVIDIA vGPU VFIO variant driver Zhi Wang
2026-09-05 8:11 ` [PATCH 13/13] gpu: nova-core: reserve the 48-VM WPR2 heap Zhi Wang
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
The NVIDIA vGPU VFIO variant driver needs nova-core to create, reset,
and destroy firmware-backed vGPU instances on behalf of virtual
functions.
Add an internal C interface for these operations and implement its entry
points in Rust. The open operation queries the profile assigned to the
VF and creates and activates an instance. Close and reset run the
corresponding teardown and reinitialization sequences.
Keep registry locking, allocation, activation, publication, and rollback
inside VgpuManager so the VFIO caller does not coordinate manager state.
Extend the existing C export shim to publish the three GPL-only symbols
in the NOVA_CORE_VGPU namespace, and use the Rust #[export] attribute to
verify their signatures against the C declarations at build time.
Co-developed-by: Alok Kumar <alkumar@nvidia.com>
Signed-off-by: Alok Kumar <alkumar@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/gpu.rs | 55 ++++-
drivers/gpu/nova-core/nova_core_exports.c | 5 +
drivers/gpu/nova-core/vgpu/fw/commands.rs | 1 -
drivers/gpu/nova-core/vgpu/instance.rs | 145 ++++++++++-
drivers/gpu/nova-core/vgpu/mod.rs | 11 +-
drivers/gpu/nova-core/vgpu/vfio.rs | 282 ++++++++++++++++++++++
include/drm/nvidia_vgpu.h | 28 +++
rust/bindings/bindings_helper.h | 1 +
8 files changed, 504 insertions(+), 24 deletions(-)
create mode 100644 drivers/gpu/nova-core/vgpu/vfio.rs
create mode 100644 include/drm/nvidia_vgpu.h
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index f88be4ca6ea5..b0b1f26c47ba 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -1,6 +1,9 @@
// SPDX-License-Identifier: GPL-2.0
-use core::ops::Range;
+use core::{
+ num::NonZero,
+ ops::Range, //
+};
use kernel::{
device,
@@ -9,6 +12,7 @@
fmt,
gpu::buddy::GpuBuddyParams,
io::Io,
+ new_mutex,
num::Bounded,
pci,
prelude::*,
@@ -17,7 +21,10 @@
SizeConstants,
SZ_4K, //
},
- sync::Arc,
+ sync::{
+ Arc,
+ Mutex, //
+ },
};
use crate::{
@@ -318,7 +325,8 @@ pub(crate) struct Gpu<'gpu> {
///
/// Must be kept declared *before* `gsp_resources`, so that its components are dropped while
/// the GSP is still operational.
- mm: GpuMm<'gpu>,
+ #[pin]
+ mm: Mutex<GpuMm<'gpu>>,
/// BAR1 user interface for CPU access to GPU virtual memory.
#[pin]
bar_user: BarUser<'gpu>,
@@ -377,11 +385,30 @@ pub(crate) fn cmdq(&self) -> Arc<Cmdq> {
}
/// Returns the firmware build identifier, if one was reported.
- #[expect(dead_code)]
pub(crate) fn build_id(&self) -> Option<firmware::BuildId> {
self.gsp_resources.gsp.build_id()
}
+ pub(crate) fn vgpu_manager(&self) -> &VgpuManager<'gpu> {
+ &self.vgpu
+ }
+
+ pub(crate) fn vgpu_total_vfs(&self) -> Option<NonZero<u16>> {
+ self.vgpu.total_vfs()
+ }
+
+ pub(crate) fn mm(&self) -> &Mutex<GpuMm<'gpu>> {
+ &self.mm
+ }
+
+ pub(crate) fn bar_user(&self) -> &BarUser<'gpu> {
+ &self.bar_user
+ }
+
+ pub(crate) fn bar0(&self) -> Bar0<'gpu> {
+ self.gsp_resources.bar
+ }
+
pub(crate) fn new(
pdev: &'gpu pci::Device<device::Core<'_>>,
bar: Bar0<'gpu>,
@@ -505,7 +532,7 @@ pub(crate) fn new(
},
// Create GPU memory manager owning memory management resources.
- mm: {
+ mm <- {
let info = &gsp_resources.boot_result.static_info;
let usable_vram = info.usable_fb_regions.first().ok_or(ENODEV)?;
let buddy_params = GpuBuddyParams {
@@ -514,12 +541,15 @@ pub(crate) fn new(
chunk_size: Alignment::new::<SZ_4K>(),
};
- GpuMm::new(
- bar,
- gsp_resources.spec.chipset,
- buddy_params,
- VramAddress::from_raw(info.total_fb_end),
- )?
+ new_mutex!(
+ GpuMm::new(
+ bar,
+ gsp_resources.spec.chipset,
+ buddy_params,
+ VramAddress::from_raw(info.total_fb_end),
+ )?,
+ "nova-core::gpu-mm",
+ )
},
// Create BAR1 user interface for CPU access to GPU virtual memory.
@@ -550,10 +580,11 @@ pub(crate) fn run_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bou
.boot_result
.static_info;
let regions = &info.usable_fb_regions;
+ let mut mm = this.mm.lock();
if let Err(err) = crate::mm::selftest::run(
dev,
- this.mm,
+ &mut mm,
regions,
this.bar_user.as_ref().get_ref(),
info.bar1_pde_base,
diff --git a/drivers/gpu/nova-core/nova_core_exports.c b/drivers/gpu/nova-core/nova_core_exports.c
index 6e80ca9792ee..cda87dfbfcdd 100644
--- a/drivers/gpu/nova-core/nova_core_exports.c
+++ b/drivers/gpu/nova-core/nova_core_exports.c
@@ -8,8 +8,13 @@
* dependencies natively.
*/
+#include <drm/nvidia_vgpu.h>
#include <linux/export.h>
+EXPORT_SYMBOL_NS_GPL(nvidia_vgpu_open, "NOVA_CORE_VGPU");
+EXPORT_SYMBOL_NS_GPL(nvidia_vgpu_close, "NOVA_CORE_VGPU");
+EXPORT_SYMBOL_NS_GPL(nvidia_vgpu_reset, "NOVA_CORE_VGPU");
+
#define EXPORT_SYMBOL_RUST_GPL(sym) extern int sym; EXPORT_SYMBOL_GPL(sym)
#include "exports_nova_core_generated.h"
diff --git a/drivers/gpu/nova-core/vgpu/fw/commands.rs b/drivers/gpu/nova-core/vgpu/fw/commands.rs
index 8527e2c430bf..d378f2615084 100644
--- a/drivers/gpu/nova-core/vgpu/fw/commands.rs
+++ b/drivers/gpu/nova-core/vgpu/fw/commands.rs
@@ -23,7 +23,6 @@ pub(crate) enum RpcResponse {
pub(crate) enum RpcMessage {
VersionNegotiation = bindings::MESSAGE_NV_VGPU_CPU_RPC_MSG_VERSION_NEGOTIATION,
SetupConfigParamsAndInit = bindings::MESSAGE_NV_VGPU_CPU_RPC_MSG_SETUP_CONFIG_PARAMS_AND_INIT,
- #[expect(dead_code)]
Reset = bindings::MESSAGE_NV_VGPU_CPU_RPC_MSG_RESET,
UpdateBmeState = bindings::MESSAGE_NV_VGPU_CPU_RPC_MSG_UPDATE_BME_STATE,
}
diff --git a/drivers/gpu/nova-core/vgpu/instance.rs b/drivers/gpu/nova-core/vgpu/instance.rs
index ed715951e92d..65abe45b8a2d 100644
--- a/drivers/gpu/nova-core/vgpu/instance.rs
+++ b/drivers/gpu/nova-core/vgpu/instance.rs
@@ -9,7 +9,8 @@
prelude::*,
ptr::Alignment,
sizes::SizeConstants,
- str::CString, //
+ str::CString,
+ sync::Mutex, //
};
use crate::{
@@ -41,7 +42,8 @@
consts::gmc,
fw::{
CommBufferRegion,
- MappedPluginLogBuffers, //
+ MappedPluginLogBuffers,
+ RpcMessage, //
},
log::VgpuLogBuffers,
plugin_rpc::{
@@ -92,6 +94,22 @@ pub(crate) const fn vgpu_type_id(&self) -> u32 {
self.vgpu_type_id
}
+ pub(crate) const fn bar1_length(&self) -> u64 {
+ self.bar1_length
+ }
+
+ pub(crate) const fn pci_dev_id(&self) -> u32 {
+ self.pci_dev_id
+ }
+
+ pub(crate) const fn pci_subsys_id(&self) -> u32 {
+ self.pci_subsys_id
+ }
+
+ pub(crate) const fn fb_length(&self) -> u64 {
+ self.fb_length
+ }
+
fn from_properties(properties: &VgpuProperties) -> Self {
let mut name = [0; 64];
let name_len = properties.name.len().min(name.len());
@@ -127,7 +145,6 @@ fn from_properties(properties: &VgpuProperties) -> Self {
/// Field ordering is load-bearing for drop: `debugfs_logs` must be declared
/// before `plugin_rpc` so that debugfs entries are removed (and in-progress
/// readers drained) before the underlying `Bar1Map` is destroyed.
-#[expect(dead_code)]
pub(crate) struct VgpuInstance<'gpu> {
pub(crate) gfid: Gfid,
pub(crate) dbdf: Dbdf,
@@ -139,6 +156,7 @@ pub(crate) struct VgpuInstance<'gpu> {
pub(crate) vram_slot: VgpuVramSlot,
debugfs_logs: Option<Pin<KBox<debugfs::Scope<VgpuLogBuffers>>>>,
pub(crate) plugin_rpc: PluginRpc<'gpu>,
+ active: bool,
}
impl<'gpu> VgpuInstance<'gpu> {
@@ -221,7 +239,6 @@ pub(crate) struct InstanceInfo {
pub(crate) vm_pid: u32,
}
-#[expect(dead_code)]
impl InstanceInfo {
pub(crate) const fn new(gfid: Gfid, dbdf: Dbdf, vgpu_type: VgpuType, vm_pid: u32) -> Self {
Self {
@@ -273,7 +290,6 @@ pub(crate) struct VgpuInstances<'gpu> {
vram_slots: Option<VgpuVramSlotAllocator>,
}
-#[expect(dead_code)]
impl<'gpu> VgpuInstances<'gpu> {
pub(crate) const fn new() -> Self {
Self {
@@ -444,6 +460,7 @@ pub(crate) fn allocate_instance(
vram_slot,
debugfs_logs: None,
plugin_rpc: PluginRpc::new(comm),
+ active: false,
};
match self.instances.push_within_capacity(instance) {
Ok(()) => Ok(gfid),
@@ -466,6 +483,31 @@ pub(crate) fn allocate_instance(
}
}
+ /// Reset an active instance and scrub its guest framebuffer.
+ pub(crate) fn reset_instance(
+ &mut self,
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ gfid: Gfid,
+ ) -> Result {
+ let instance = self
+ .instances
+ .iter_mut()
+ .find(|instance| instance.gfid == gfid)
+ .ok_or(ENOENT)?;
+ if !instance.active {
+ return Err(EBUSY);
+ }
+
+ instance
+ .plugin_rpc
+ .rpc_call(dev, bar, gfid, RpcMessage::Reset, &[])?;
+ instance.scrub_guest_fb(dev, cmdq, bar, bar_user, mm)
+ }
+
/// Shut down an instance, scrub its guest FB, and release its reservations.
pub(crate) fn destroy_instance(
&mut self,
@@ -483,6 +525,7 @@ pub(crate) fn destroy_instance(
.ok_or(ENOENT)?;
shutdown(dev, cmdq, bar, gfid)?;
+ self.instances[index].active = false;
self.instances[index].scrub_guest_fb(dev, cmdq, bar, bar_user, mm)?;
self.instances[index].release_ceutils(dev, cmdq, bar)?;
cleanup(dev, cmdq, bar, gfid)?;
@@ -494,7 +537,6 @@ pub(crate) fn destroy_instance(
}
/// Query the vGPU type assigned to a VF by its DBDF.
-#[expect(dead_code)]
pub(crate) fn query_assigned_vf_type(cmdq: &Cmdq, bar: Bar0<'_>, dbdf: Dbdf) -> Result<u32> {
let request = u64::from(dbdf.into_raw()).to_le_bytes();
let response =
@@ -507,7 +549,6 @@ pub(crate) fn query_assigned_vf_type(cmdq: &Cmdq, bar: Bar0<'_>, dbdf: Dbdf) ->
}
/// Query and decode one vGPU type using the typed NVKV schema.
-#[expect(dead_code)]
pub(crate) fn query_vgpu_type(cmdq: &Cmdq, bar: Bar0<'_>, type_id: u32) -> Result<VgpuType> {
let response = cmdq.send_gmc_and_receive(
bar,
@@ -531,8 +572,7 @@ pub(crate) fn query_vgpu_type(cmdq: &Cmdq, bar: Bar0<'_>, type_id: u32) -> Resul
/// Ask GSP to create the plugin task, wait for its BAR1 ready marker,
/// initialize the shared RPC buffers, negotiate the protocol, send the
/// instance configuration, and enable bus mastering.
-#[expect(dead_code)]
-pub(crate) fn activate_instance(
+fn activate_instance(
dev: &device::Device<device::Bound>,
cmdq: &Cmdq,
bar: Bar0<'_>,
@@ -576,3 +616,90 @@ pub(crate) fn activate_instance(
Ok(())
}
+
+/// Activate an instance already owned by the live-instance registry.
+///
+/// If activation fails, attempt full teardown before returning the original
+/// error.
+#[expect(clippy::too_many_arguments)]
+fn activate_registered_instance<'gpu>(
+ instances: &mut VgpuInstances<'gpu>,
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ bar_user: &BarUser<'gpu>,
+ mm: &mut GpuMm<'_>,
+ gfid: Gfid,
+ fifo_engine_list: &FifoEngineList,
+ chipset: Chipset,
+ build_id: Option<&BuildId>,
+) -> Result {
+ let index = instances
+ .instances
+ .iter()
+ .position(|instance| instance.gfid == gfid)
+ .ok_or(EIO)?;
+ let activation_result = activate_instance(
+ dev,
+ cmdq,
+ bar,
+ &mut instances.instances[index],
+ fifo_engine_list,
+ chipset,
+ build_id,
+ );
+
+ if let Err(original_error) = activation_result {
+ if let Err(cleanup_error) = instances.destroy_instance(dev, cmdq, bar, bar_user, mm, gfid) {
+ dev_err!(
+ dev,
+ "vgpu_open: cleanup failed for gfid={} after activation error {:?}: {:?}\n",
+ gfid.0,
+ original_error,
+ cleanup_error,
+ );
+ }
+ return Err(original_error);
+ }
+
+ instances.instances[index].active = true;
+ Ok(())
+}
+
+impl<'gpu> VgpuManager<'gpu> {
+ /// Allocate, register, and activate a vGPU instance.
+ ///
+ /// Keep the registry locked from allocation through activation or rollback
+ /// so duplicate checks and profile limits remain stable.
+ #[expect(clippy::too_many_arguments)]
+ pub(crate) fn create_instance(
+ &self,
+ dev: &device::Device<device::Bound>,
+ cmdq: &Cmdq,
+ bar: Bar0<'_>,
+ bar_user: &BarUser<'gpu>,
+ mm: &Mutex<GpuMm<'gpu>>,
+ info: InstanceInfo,
+ chipset: Chipset,
+ build_id: Option<&BuildId>,
+ ) -> Result {
+ let fifo_engine_list = self.fifo_engine_list()?;
+ let mut instances = self.instances().lock();
+ // Global vGPU lock order: instances -> MM -> BAR-user VMM.
+ let mut mm = mm.lock();
+ let gfid = instances.allocate_instance(dev, cmdq, bar, bar_user, &mut mm, self, info)?;
+
+ activate_registered_instance(
+ &mut instances,
+ dev,
+ cmdq,
+ bar,
+ bar_user,
+ &mut mm,
+ gfid,
+ fifo_engine_list,
+ chipset,
+ build_id,
+ )
+ }
+}
diff --git a/drivers/gpu/nova-core/vgpu/mod.rs b/drivers/gpu/nova-core/vgpu/mod.rs
index 61f799c2748e..34ab7eaed55b 100644
--- a/drivers/gpu/nova-core/vgpu/mod.rs
+++ b/drivers/gpu/nova-core/vgpu/mod.rs
@@ -9,6 +9,7 @@
pub(crate) mod log;
pub(crate) mod plugin_rpc;
pub(crate) mod scrubber;
+mod vfio;
pub(crate) use self::instance::VgpuInstances;
@@ -124,6 +125,14 @@ pub(crate) fn state(&self) -> VgpuState {
self.state
}
+ /// Returns the number of VFs available to an enabled vGPU boot.
+ pub(crate) fn total_vfs(&self) -> Option<NonZero<u16>> {
+ match self.state {
+ VgpuState::Disabled => None,
+ VgpuState::Enabled { total_vfs } => Some(total_vfs),
+ }
+ }
+
/// Initializes the runtime parameters returned by GSP_INIT.
pub(crate) fn init(
self: Pin<&mut Self>,
@@ -140,7 +149,6 @@ pub(crate) fn init(
}
/// Returns the live-instance registry.
- #[expect(dead_code)]
pub(crate) fn instances(&self) -> &Mutex<VgpuInstances<'gpu>> {
&self.instances
}
@@ -156,7 +164,6 @@ pub(crate) const fn total_channels(&self) -> Option<u32> {
}
/// Returns the ordered FIFO engine list provided by GSP_INIT.
- #[expect(dead_code)]
pub(crate) fn fifo_engine_list(&self) -> Result<&FifoEngineList> {
self.fifo_engine_list.as_ref().ok_or(ENODEV)
}
diff --git a/drivers/gpu/nova-core/vgpu/vfio.rs b/drivers/gpu/nova-core/vgpu/vfio.rs
new file mode 100644
index 000000000000..967f74ba8ea4
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/vfio.rs
@@ -0,0 +1,282 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+//! FFI exports for the VFIO variant driver.
+
+use kernel::{
+ device,
+ pci,
+ prelude::*, //
+};
+
+use crate::{
+ driver::NovaCore,
+ gsp::commands::Dbdf,
+ vgpu::instance::{
+ query_assigned_vf_type,
+ query_vgpu_type,
+ Gfid,
+ InstanceInfo,
+ VgpuType, //
+ },
+};
+
+/// Transparent wrapper over the C `struct nvidia_vgpu_type_info`.
+#[repr(transparent)]
+pub(crate) struct VgpuTypeInfo(kernel::bindings::nvidia_vgpu_type_info);
+
+impl VgpuTypeInfo {
+ fn from_vgpu_type(vgpu_type: &VgpuType) -> Self {
+ Self(kernel::bindings::nvidia_vgpu_type_info {
+ pci_dev_id: vgpu_type.pci_dev_id(),
+ pci_subsys_id: vgpu_type.pci_subsys_id(),
+ bar1_length: vgpu_type.bar1_length(),
+ })
+ }
+}
+
+/// Run `f` with the nova-core driver data and bound device for a PF.
+///
+/// The higher-ranked callback prevents references reconstructed from the raw
+/// PCI device and its driver data from escaping this call.
+///
+/// # Safety
+///
+/// If `pf_pdev` is non-null, it must point to a live physical PCI function and
+/// remain valid for the duration of this call. The function must remain bound
+/// to nova-core, without concurrent unbind, for the duration of this call.
+unsafe fn with_nova_core<R>(
+ pf_pdev: *mut kernel::bindings::pci_dev,
+ gfid: u32,
+ f: impl for<'a> FnOnce(Pin<&'a NovaCore<'a>>, &'a device::Device<device::Bound>) -> Result<R>,
+) -> Result<R> {
+ if pf_pdev.is_null() {
+ return Err(EINVAL);
+ }
+
+ // SAFETY: The caller guarantees that `pf_pdev` points to a live PCI
+ // device that remains bound for this call. `pci::Device` is transparent
+ // over `bindings::pci_dev`.
+ let pf: &pci::Device<device::Bound> = unsafe { &*pf_pdev.cast() };
+ if pf.is_virtfn() || !pf.is_physfn() {
+ return Err(EINVAL);
+ }
+
+ // Before interpreting drvdata as `NovaCore`, verify that this PF is still
+ // bound to the nova-core PCI driver. `managed_sriov` keeps the PF bound for
+ // the lifetime of its VFs.
+ // SAFETY: `pf_pdev` is valid and its `driver` pointer, when non-null,
+ // remains valid while the device is bound.
+ let driver = unsafe { (*pf_pdev).driver };
+ if driver.is_null()
+ // SAFETY: `driver` was checked for null above.
+ || !unsafe { (*driver).managed_sriov }
+ // SAFETY: `driver` was checked for null above.
+ || unsafe { (*driver).name } != crate::MODULE_NAME.as_char_ptr()
+ // SAFETY: `driver` was checked for null above.
+ || unsafe { (*driver).driver.owner } != crate::THIS_MODULE.as_ptr()
+ {
+ return Err(ENODEV);
+ }
+
+ let pf_dev: &device::Device<device::Bound> = pf.as_ref();
+ // SAFETY: `pf_pdev` is valid, so its embedded device is valid too.
+ let drvdata =
+ unsafe { kernel::bindings::dev_get_drvdata(core::ptr::addr_of_mut!((*pf_pdev).dev)) };
+ if drvdata.is_null() {
+ return Err(ENODEV);
+ }
+
+ // SAFETY: The driver identity check above establishes that drvdata was
+ // installed by nova-core as `NovaCore`. The caller guarantees that the PF
+ // cannot be unbound during this call, and PCI driver data stores the
+ // pointer returned by `Pin<KBox<NovaCore>>::into_foreign()`. Lifetimes do
+ // not affect layout. The callback's HRTB prevents the reconstructed
+ // reference, including NovaCore's bound-device lifetime, from escaping.
+ let nova_core = unsafe { Pin::new_unchecked(&*drvdata.cast::<NovaCore<'_>>()) };
+
+ let total_vfs = nova_core.gpu.vgpu_total_vfs().ok_or(ENODEV)?;
+ if gfid == 0 || gfid > u32::from(total_vfs.get()) {
+ return Err(EINVAL);
+ }
+
+ f(nova_core, pf_dev)
+}
+
+fn nvidia_vgpu_open_inner<'a>(
+ nova_core: Pin<&'a NovaCore<'a>>,
+ dev: &'a device::Device<device::Bound>,
+ gfid: u32,
+ dbdf: u32,
+ vm_pid: u32,
+) -> Result<VgpuTypeInfo> {
+ let gpu = &nova_core.gpu;
+ let gfid = Gfid(gfid);
+ let dbdf = Dbdf::from_raw(dbdf);
+
+ dev_dbg!(
+ dev,
+ "vgpu_open: gfid={} dbdf={:#x}\n",
+ gfid.0,
+ dbdf.into_raw()
+ );
+
+ let bar = gpu.bar0();
+ let cmdq = gpu.cmdq();
+ let vgpu = gpu.vgpu_manager();
+
+ let type_id = query_assigned_vf_type(&cmdq, bar, dbdf)?;
+ dev_dbg!(
+ dev,
+ "vgpu_open: gfid={} assigned type_id={}\n",
+ gfid.0,
+ type_id
+ );
+
+ let vgpu_type = query_vgpu_type(&cmdq, bar, type_id)?;
+ dev_dbg!(
+ dev,
+ "vgpu_open: gfid={} vgpu_type={} fb_length={:#x}\n",
+ gfid.0,
+ vgpu_type.vgpu_type_id(),
+ vgpu_type.fb_length()
+ );
+
+ let type_info = VgpuTypeInfo::from_vgpu_type(&vgpu_type);
+ let build_id = gpu.build_id();
+ vgpu.create_instance(
+ dev,
+ &cmdq,
+ bar,
+ gpu.bar_user(),
+ gpu.mm(),
+ InstanceInfo::new(gfid, dbdf, vgpu_type, vm_pid),
+ gpu.chipset(),
+ build_id.as_ref(),
+ )?;
+
+ Ok(type_info)
+}
+
+fn nvidia_vgpu_close_inner<'a>(
+ nova_core: Pin<&'a NovaCore<'a>>,
+ dev: &'a device::Device<device::Bound>,
+ gfid: u32,
+) -> Result {
+ let gpu = &nova_core.gpu;
+ let gfid = Gfid(gfid);
+
+ dev_dbg!(dev, "vgpu_close: gfid={}\n", gfid.0);
+
+ let cmdq = gpu.cmdq();
+ let mut instances = gpu.vgpu_manager().instances().lock();
+ let mut mm = gpu.mm().lock();
+ let result = instances.destroy_instance(dev, &cmdq, gpu.bar0(), gpu.bar_user(), &mut mm, gfid);
+ if let Err(error) = result {
+ dev_err!(dev, "vgpu_close: gfid={} failed: {:?}\n", gfid.0, error);
+ }
+ result
+}
+
+fn nvidia_vgpu_reset_inner<'a>(
+ nova_core: Pin<&'a NovaCore<'a>>,
+ dev: &'a device::Device<device::Bound>,
+ gfid: u32,
+) -> Result {
+ let gpu = &nova_core.gpu;
+ let gfid = Gfid(gfid);
+
+ dev_dbg!(dev, "vgpu_reset: gfid={}\n", gfid.0);
+
+ let cmdq = gpu.cmdq();
+ let mut instances = gpu.vgpu_manager().instances().lock();
+ let mut mm = gpu.mm().lock();
+ instances.reset_instance(dev, &cmdq, gpu.bar0(), gpu.bar_user(), &mut mm, gfid)?;
+
+ dev_dbg!(dev, "vgpu_reset: gfid={} done\n", gfid.0);
+ Ok(())
+}
+
+/// # Safety
+///
+/// If `pf_pdev` is non-null, it must point to a live physical PCI function
+/// that remains bound to nova-core, without concurrent unbind, for the
+/// duration of this call. `gfid` is the Guest Function ID (VF index + 1).
+/// `dbdf` is the VF's Domain:Bus:Device.Function encoded as
+/// `(domain << 16) | (bus << 8) | devfn`. `vm_pid` is the thread-group ID of
+/// the userspace VM process. If `type_info` is non-null, it must point to
+/// aligned, writable storage for a `struct nvidia_vgpu_type_info`, and no
+/// other thread may access that storage for the duration of the write.
+#[export]
+unsafe extern "C" fn nvidia_vgpu_open(
+ pf_pdev: *mut kernel::bindings::pci_dev,
+ gfid: core::ffi::c_uint,
+ dbdf: core::ffi::c_uint,
+ vm_pid: core::ffi::c_uint,
+ type_info: *mut kernel::bindings::nvidia_vgpu_type_info,
+) -> core::ffi::c_int {
+ if type_info.is_null() {
+ return EINVAL.to_errno();
+ }
+
+ // SAFETY: The caller upholds the exported function's contract. The HRTB
+ // callback confines all references derived from `pf_pdev` to this call.
+ let result = unsafe {
+ with_nova_core(pf_pdev, gfid, |nova_core, dev| {
+ nvidia_vgpu_open_inner(nova_core, dev, gfid, dbdf, vm_pid)
+ })
+ };
+
+ match result {
+ Ok(info) => {
+ // SAFETY: `type_info` was checked for null above and the caller
+ // guarantees that it points to writable storage.
+ unsafe { type_info.write(info.0) };
+ 0
+ }
+ Err(error) => error.to_errno(),
+ }
+}
+
+/// # Safety
+///
+/// If `pf_pdev` is non-null, it must point to a live physical PCI function
+/// that remains bound to nova-core, without concurrent unbind, for the
+/// duration of this call. `gfid` must identify one of that PF's VFs.
+#[export]
+unsafe extern "C" fn nvidia_vgpu_close(
+ pf_pdev: *mut kernel::bindings::pci_dev,
+ gfid: core::ffi::c_uint,
+) {
+ // SAFETY: The caller upholds the exported function's contract. The HRTB
+ // callback confines all references derived from `pf_pdev` to this call.
+ let _ = unsafe {
+ with_nova_core(pf_pdev, gfid, |nova_core, dev| {
+ nvidia_vgpu_close_inner(nova_core, dev, gfid)
+ })
+ };
+}
+
+/// # Safety
+///
+/// If `pf_pdev` is non-null, it must point to a live physical PCI function
+/// that remains bound to nova-core, without concurrent unbind, for the
+/// duration of this call. `gfid` must identify one of that PF's VFs.
+#[export]
+unsafe extern "C" fn nvidia_vgpu_reset(
+ pf_pdev: *mut kernel::bindings::pci_dev,
+ gfid: core::ffi::c_uint,
+) -> core::ffi::c_int {
+ // SAFETY: The caller upholds the exported function's contract. The HRTB
+ // callback confines all references derived from `pf_pdev` to this call.
+ let result = unsafe {
+ with_nova_core(pf_pdev, gfid, |nova_core, dev| {
+ nvidia_vgpu_reset_inner(nova_core, dev, gfid)
+ })
+ };
+
+ match result {
+ Ok(()) => 0,
+ Err(error) => error.to_errno(),
+ }
+}
diff --git a/include/drm/nvidia_vgpu.h b/include/drm/nvidia_vgpu.h
new file mode 100644
index 000000000000..d49bb57db6f4
--- /dev/null
+++ b/include/drm/nvidia_vgpu.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#ifndef __DRM_NVIDIA_VGPU_H__
+#define __DRM_NVIDIA_VGPU_H__
+
+#include <linux/types.h>
+
+struct pci_dev;
+
+/**
+ * struct nvidia_vgpu_type_info - vGPU type descriptor returned by open
+ * @pci_dev_id: PCI device ID to present to the guest
+ * @pci_subsys_id: PCI subsystem ID to present to the guest
+ * @bar1_length: BAR1 aperture size in MiB
+ */
+struct nvidia_vgpu_type_info {
+ u32 pci_dev_id;
+ u32 pci_subsys_id;
+ u64 bar1_length;
+};
+
+int nvidia_vgpu_open(struct pci_dev *pf_pdev, unsigned int gfid,
+ unsigned int dbdf, unsigned int vm_pid,
+ struct nvidia_vgpu_type_info *type_info);
+void nvidia_vgpu_close(struct pci_dev *pf_pdev, unsigned int gfid);
+int nvidia_vgpu_reset(struct pci_dev *pf_pdev, unsigned int gfid);
+
+#endif /* __DRM_NVIDIA_VGPU_H__ */
diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h
index 3d0511e4ab4f..d4540e719421 100644
--- a/rust/bindings/bindings_helper.h
+++ b/rust/bindings/bindings_helper.h
@@ -37,6 +37,7 @@
#include <drm/drm_gem_shmem_helper.h>
#include <drm/drm_gpuvm.h>
#include <drm/drm_ioctl.h>
+#include <drm/nvidia_vgpu.h>
#include <kunit/test.h>
#include <linux/auxiliary_bus.h>
#include <linux/bitmap.h>
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 12/13] vfio/nvidia-vgpu: add the NVIDIA vGPU VFIO variant driver
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
` (10 preceding siblings ...)
2026-09-05 8:11 ` [PATCH 11/13] gpu: nova-core: vgpu: export lifecycle operations to VFIO Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
2026-09-09 3:00 ` Alex Williamson
2026-09-05 8:11 ` [PATCH 13/13] gpu: nova-core: reserve the 48-VM WPR2 heap Zhi Wang
12 siblings, 1 reply; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang, kvm
NVIDIA vGPU VFs require their open, reset, and close lifecycle to be
coordinated with the PF-side nova-core driver.
Add a VFIO PCI variant driver that binds NVIDIA devices only through
driver_override and rejects non-VFs. Delegate instance lifecycle
operations to nova-core, present the firmware-selected device and
subsystem IDs in configuration-space reads, and adjust the reported BAR1
aperture to the assigned profile. Use vfio-pci-core for the remaining
VFIO operations.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/vfio/pci/Kconfig | 2 +
drivers/vfio/pci/Makefile | 2 +
drivers/vfio/pci/nvidia-vgpu/Kconfig | 16 ++
drivers/vfio/pci/nvidia-vgpu/Makefile | 2 +
drivers/vfio/pci/nvidia-vgpu/main.c | 253 ++++++++++++++++++++++++++
5 files changed, 275 insertions(+)
create mode 100644 drivers/vfio/pci/nvidia-vgpu/Kconfig
create mode 100644 drivers/vfio/pci/nvidia-vgpu/Makefile
create mode 100644 drivers/vfio/pci/nvidia-vgpu/main.c
diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig
index 296bf01e185e..b48d8d1af42a 100644
--- a/drivers/vfio/pci/Kconfig
+++ b/drivers/vfio/pci/Kconfig
@@ -74,4 +74,6 @@ source "drivers/vfio/pci/qat/Kconfig"
source "drivers/vfio/pci/xe/Kconfig"
+source "drivers/vfio/pci/nvidia-vgpu/Kconfig"
+
endmenu
diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile
index 6138f1bf241d..f3498e541555 100644
--- a/drivers/vfio/pci/Makefile
+++ b/drivers/vfio/pci/Makefile
@@ -24,3 +24,5 @@ obj-$(CONFIG_NVGRACE_GPU_VFIO_PCI) += nvgrace-gpu/
obj-$(CONFIG_QAT_VFIO_PCI) += qat/
obj-$(CONFIG_XE_VFIO_PCI) += xe/
+
+obj-$(CONFIG_NVIDIA_VGPU_VFIO_PCI) += nvidia-vgpu/
diff --git a/drivers/vfio/pci/nvidia-vgpu/Kconfig b/drivers/vfio/pci/nvidia-vgpu/Kconfig
new file mode 100644
index 000000000000..098822d32380
--- /dev/null
+++ b/drivers/vfio/pci/nvidia-vgpu/Kconfig
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config NVIDIA_VGPU_VFIO_PCI
+ tristate "VFIO support for the NVIDIA vGPU"
+ depends on NOVA_CORE && PCI_IOV
+ select VFIO_PCI_CORE
+ help
+ This option enables VFIO (Virtual Function I/O) support for
+ NVIDIA virtual GPUs (vGPU). It allows the assignment of a virtual
+ GPU instance to userspace applications via VFIO, typically used
+ with hypervisors such as KVM and device emulators like QEMU.
+
+ The NVIDIA vGPU allows a physical GPU to be partitioned into
+ multiple virtual GPUs, each of which can be passed to a virtual
+ machine as a PCI device using the standard VFIO infrastructure.
+
+ If you don't know what to do here, say N.
diff --git a/drivers/vfio/pci/nvidia-vgpu/Makefile b/drivers/vfio/pci/nvidia-vgpu/Makefile
new file mode 100644
index 000000000000..193cc801a081
--- /dev/null
+++ b/drivers/vfio/pci/nvidia-vgpu/Makefile
@@ -0,0 +1,2 @@
+obj-$(CONFIG_NVIDIA_VGPU_VFIO_PCI) += nvidia-vgpu-vfio-pci.o
+nvidia-vgpu-vfio-pci-y := main.o
diff --git a/drivers/vfio/pci/nvidia-vgpu/main.c b/drivers/vfio/pci/nvidia-vgpu/main.c
new file mode 100644
index 000000000000..d8626644f952
--- /dev/null
+++ b/drivers/vfio/pci/nvidia-vgpu/main.c
@@ -0,0 +1,253 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <linux/module.h>
+#include <linux/overflow.h>
+#include <linux/pci.h>
+#include <linux/pid.h>
+#include <linux/vfio_pci_core.h>
+#include <drm/nvidia_vgpu.h>
+
+static int nvidia_vgpu_fb_bar_index(struct pci_dev *pdev)
+{
+ if (pci_resource_flags(pdev, 0) & IORESOURCE_MEM_64)
+ return 2;
+ return 1;
+}
+
+struct nvidia_vgpu_pci_core_device {
+ struct vfio_pci_core_device core_device;
+ struct nvidia_vgpu_type_info type_info;
+ unsigned int gfid;
+};
+
+static inline unsigned int nvidia_vgpu_vf_dbdf(struct pci_dev *vf)
+{
+ return ((u32)pci_domain_nr(vf->bus) << 16) | pci_dev_id(vf);
+}
+
+static int nvidia_vgpu_open_device(struct vfio_device *core_vdev)
+{
+ struct nvidia_vgpu_pci_core_device *nvdev = container_of(
+ core_vdev, struct nvidia_vgpu_pci_core_device, core_device.vdev);
+ struct pci_dev *vf = to_pci_dev(core_vdev->dev);
+ struct nvidia_vgpu_type_info type_info;
+ int ret;
+
+ if (!vf->is_virtfn)
+ return -ENODEV;
+
+ ret = vfio_pci_core_enable(&nvdev->core_device);
+ if (ret)
+ return ret;
+
+ ret = nvidia_vgpu_open(pci_physfn(vf), nvdev->gfid,
+ nvidia_vgpu_vf_dbdf(vf), task_tgid_nr(current),
+ &type_info);
+ if (ret) {
+ vfio_pci_core_disable(&nvdev->core_device);
+ return ret;
+ }
+
+ nvdev->type_info = type_info;
+ pci_dbg(vf, "vgpu open: dev_id=0x%x subsys_id=0x%x bar1_length=0x%llx\n",
+ type_info.pci_dev_id, type_info.pci_subsys_id,
+ type_info.bar1_length);
+ vfio_pci_core_finish_enable(&nvdev->core_device);
+ return 0;
+}
+
+static void nvidia_vgpu_close_device(struct vfio_device *core_vdev)
+{
+ struct nvidia_vgpu_pci_core_device *nvdev = container_of(
+ core_vdev, struct nvidia_vgpu_pci_core_device, core_device.vdev);
+ struct pci_dev *vf = to_pci_dev(core_vdev->dev);
+
+ nvidia_vgpu_close(pci_physfn(vf), nvdev->gfid);
+ vfio_pci_core_close_device(core_vdev);
+}
+
+static ssize_t nvidia_vgpu_pci_read_config(struct vfio_device *core_vdev,
+ char __user *buf, size_t count,
+ loff_t *ppos)
+{
+ struct nvidia_vgpu_pci_core_device *nvdev = container_of(
+ core_vdev, struct nvidia_vgpu_pci_core_device, core_device.vdev);
+ struct nvidia_vgpu_type_info *ti = &nvdev->type_info;
+ loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK;
+ size_t register_offset;
+ loff_t copy_offset;
+ size_t copy_count;
+ __le16 val16;
+ int ret;
+
+ ret = vfio_pci_core_read(core_vdev, buf, count, ppos);
+ if (ret < 0)
+ return ret;
+
+ if (vfio_pci_core_range_intersect_range(pos, count, PCI_DEVICE_ID,
+ sizeof(val16), ©_offset,
+ ©_count, ®ister_offset)) {
+ val16 = cpu_to_le16(ti->pci_dev_id);
+ if (copy_to_user(buf + copy_offset,
+ (void *)&val16 + register_offset, copy_count))
+ return -EFAULT;
+ }
+
+ if (vfio_pci_core_range_intersect_range(pos, count, PCI_SUBSYSTEM_ID,
+ sizeof(val16), ©_offset,
+ ©_count, ®ister_offset)) {
+ val16 = cpu_to_le16(ti->pci_subsys_id);
+ if (copy_to_user(buf + copy_offset,
+ (void *)&val16 + register_offset, copy_count))
+ return -EFAULT;
+ }
+
+ return count;
+}
+
+static ssize_t nvidia_vgpu_pci_read(struct vfio_device *core_vdev,
+ char __user *buf, size_t count,
+ loff_t *ppos)
+{
+ unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
+
+ if (index == VFIO_PCI_CONFIG_REGION_INDEX)
+ return nvidia_vgpu_pci_read_config(core_vdev, buf, count, ppos);
+
+ return vfio_pci_core_read(core_vdev, buf, count, ppos);
+}
+
+static int nvidia_vgpu_bar1_size(struct nvidia_vgpu_pci_core_device *nvdev,
+ u64 *size)
+{
+ if (check_shl_overflow(nvdev->type_info.bar1_length, 20, size))
+ return -EOVERFLOW;
+
+ return 0;
+}
+
+static int nvidia_vgpu_get_region_info(struct vfio_device *core_vdev,
+ struct vfio_region_info *info,
+ struct vfio_info_cap *caps)
+{
+ int ret;
+
+ ret = vfio_pci_ioctl_get_region_info(core_vdev, info, caps);
+ if (ret)
+ return ret;
+
+ if (info->index == nvidia_vgpu_fb_bar_index(
+ to_pci_dev(core_vdev->dev)) && info->size) {
+ struct nvidia_vgpu_pci_core_device *nvdev = container_of(
+ core_vdev, struct nvidia_vgpu_pci_core_device,
+ core_device.vdev);
+ u64 vgpu_bar1;
+
+ ret = nvidia_vgpu_bar1_size(nvdev, &vgpu_bar1);
+ if (ret)
+ return ret;
+
+ if (vgpu_bar1 && vgpu_bar1 < info->size)
+ info->size = vgpu_bar1;
+ }
+
+ return 0;
+}
+
+static long nvidia_vgpu_pci_ioctl(struct vfio_device *core_vdev,
+ unsigned int cmd, unsigned long arg)
+{
+ if (cmd == VFIO_DEVICE_RESET) {
+ struct nvidia_vgpu_pci_core_device *nvdev = container_of(
+ core_vdev, struct nvidia_vgpu_pci_core_device,
+ core_device.vdev);
+ struct pci_dev *vf = to_pci_dev(core_vdev->dev);
+ int ret;
+
+ ret = nvidia_vgpu_reset(pci_physfn(vf), nvdev->gfid);
+ if (ret)
+ return ret;
+ }
+
+ return vfio_pci_core_ioctl(core_vdev, cmd, arg);
+}
+
+static const struct vfio_device_ops nvidia_vgpu_pci_ops = {
+ .name = "nvidia-vgpu-vfio-pci",
+ .init = vfio_pci_core_init_dev,
+ .release = vfio_pci_core_release_dev,
+ .open_device = nvidia_vgpu_open_device,
+ .close_device = nvidia_vgpu_close_device,
+ .ioctl = nvidia_vgpu_pci_ioctl,
+ .get_region_info_caps = nvidia_vgpu_get_region_info,
+ .device_feature = vfio_pci_core_ioctl_feature,
+ .read = nvidia_vgpu_pci_read,
+ .write = vfio_pci_core_write,
+ .mmap = vfio_pci_core_mmap,
+ .request = vfio_pci_core_request,
+ .match = vfio_pci_core_match,
+ .match_token_uuid = vfio_pci_core_match_token_uuid,
+ .bind_iommufd = vfio_iommufd_physical_bind,
+ .unbind_iommufd = vfio_iommufd_physical_unbind,
+ .attach_ioas = vfio_iommufd_physical_attach_ioas,
+ .detach_ioas = vfio_iommufd_physical_detach_ioas,
+};
+
+static int nvidia_vgpu_pci_probe(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ struct nvidia_vgpu_pci_core_device *nvdev;
+ int vf_id;
+ int ret;
+
+ if (!pdev->is_virtfn)
+ return -ENODEV;
+
+ vf_id = pci_iov_vf_id(pdev);
+ if (vf_id < 0)
+ return vf_id;
+
+ nvdev = vfio_alloc_device(nvidia_vgpu_pci_core_device, core_device.vdev,
+ &pdev->dev, &nvidia_vgpu_pci_ops);
+ if (IS_ERR(nvdev))
+ return PTR_ERR(nvdev);
+
+ nvdev->gfid = vf_id + 1;
+ dev_set_drvdata(&pdev->dev, &nvdev->core_device);
+ ret = vfio_pci_core_register_device(&nvdev->core_device);
+ if (ret)
+ goto out_put_vdev;
+
+ return 0;
+
+out_put_vdev:
+ vfio_put_device(&nvdev->core_device.vdev);
+ return ret;
+}
+
+static void nvidia_vgpu_pci_remove(struct pci_dev *pdev)
+{
+ struct vfio_pci_core_device *core_device = dev_get_drvdata(&pdev->dev);
+
+ vfio_pci_core_unregister_device(core_device);
+ vfio_put_device(&core_device->vdev);
+}
+
+static const struct pci_device_id nvidia_vgpu_pci_table[] = {
+ /* Placeholder: match all NVIDIA VFs (vendor 0x10de) */
+ { PCI_DRIVER_OVERRIDE_DEVICE_VFIO(PCI_VENDOR_ID_NVIDIA, PCI_ANY_ID) },
+ {}
+};
+MODULE_DEVICE_TABLE(pci, nvidia_vgpu_pci_table);
+
+static struct pci_driver nvidia_vgpu_pci_driver = {
+ .name = "nvidia-vgpu-vfio-pci",
+ .id_table = nvidia_vgpu_pci_table,
+ .probe = nvidia_vgpu_pci_probe,
+ .remove = nvidia_vgpu_pci_remove,
+ .driver_managed_dma = true,
+};
+module_pci_driver(nvidia_vgpu_pci_driver);
+
+MODULE_DESCRIPTION("NVIDIA vGPU vfio-pci driver");
+MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS("NOVA_CORE_VGPU");
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 13/13] gpu: nova-core: reserve the 48-VM WPR2 heap
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
` (11 preceding siblings ...)
2026-09-05 8:11 ` [PATCH 12/13] vfio/nvidia-vgpu: add the NVIDIA vGPU VFIO variant driver Zhi Wang
@ 2026-09-05 8:11 ` Zhi Wang
12 siblings, 0 replies; 15+ messages in thread
From: Zhi Wang @ 2026-09-05 8:11 UTC (permalink / raw)
To: dakr, acourbot
Cc: alex, jgg, yishaih, skolothumtho, kevin.tian, airlied, simona,
ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, Zhi Wang
GSP-RM needs the larger r000 vGPU WPR2 heap on devices that
advertise more than 32 VFs. The default heap is too small for the 48-VF
GB202 configuration and GSP-RM fails to start.
Select the firmware-defined 48-VM size from the detected VF count while
retaining the default size for configurations with 2 through 32 VFs.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
drivers/gpu/nova-core/fb.rs | 2 +-
drivers/gpu/nova-core/gsp/fw.rs | 7 +++++--
drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs | 1 +
3 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/nova-core/fb.rs b/drivers/gpu/nova-core/fb.rs
index 4aaa46eb65fd..b82821f2a6b1 100644
--- a/drivers/gpu/nova-core/fb.rs
+++ b/drivers/gpu/nova-core/fb.rs
@@ -306,7 +306,7 @@ fn wpr2_heap_params(chipset: Chipset, vgpu_state: VgpuState, fb_size: u64) -> Re
),
VgpuState::Enabled { total_vfs } => (
u8::try_from(total_vfs.get()).map_err(|_| EINVAL)?,
- gsp::LibosParams::vgpu_wpr_heap_size(),
+ gsp::LibosParams::vgpu_wpr_heap_size(total_vfs.get()),
),
})
}
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index e6c0fac55bad..083066287dfd 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -166,8 +166,11 @@ pub(crate) fn from_chipset(chipset: Chipset) -> &'static LibosParams {
}
/// Returns the WPR heap size to reserve when vGPU is enabled.
- pub(crate) fn vgpu_wpr_heap_size() -> u64 {
- u64::from(bindings::GSP_FW_HEAP_SIZE_VGPU_DEFAULT)
+ pub(crate) fn vgpu_wpr_heap_size(total_vfs: u16) -> u64 {
+ u64::from(match total_vfs {
+ 2..=32 => bindings::GSP_FW_HEAP_SIZE_VGPU_DEFAULT,
+ _ => r000_00::GSP_FW_HEAP_SIZE_VGPU_48VMS,
+ })
}
/// Returns the amount of memory (in bytes) to allocate for the WPR heap for a framebuffer size
diff --git a/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs b/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
index 01e84dfc4b88..83105b37f317 100644
--- a/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
+++ b/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
@@ -85,6 +85,7 @@ impl<T> ::core::cmp::Eq for __BindgenUnionField<T> {}
pub const GSP_FW_HEAP_PARAM_SIZE_PER_GB: u32 = 98304;
pub const GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE: u32 = 100663296;
pub const GSP_FW_HEAP_SIZE_VGPU_DEFAULT: u32 = 609222656;
+pub const GSP_FW_HEAP_SIZE_VGPU_48VMS: u32 = 1436549120;
pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MIN_MB: u32 = 64;
pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB: u32 = 256;
pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MIN_MB: u32 = 88;
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 12/13] vfio/nvidia-vgpu: add the NVIDIA vGPU VFIO variant driver
2026-09-05 8:11 ` [PATCH 12/13] vfio/nvidia-vgpu: add the NVIDIA vGPU VFIO variant driver Zhi Wang
@ 2026-09-09 3:00 ` Alex Williamson
0 siblings, 0 replies; 15+ messages in thread
From: Alex Williamson @ 2026-09-09 3:00 UTC (permalink / raw)
To: Zhi Wang
Cc: dakr, acourbot, jgg, yishaih, skolothumtho, kevin.tian, airlied,
simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin,
a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, cjia, smitra,
kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu,
linux-kernel, zhiwang, kvm, alex
On Sat, 5 Sep 2026 11:11:15 +0300
Zhi Wang <zhiw@nvidia.com> wrote:
> NVIDIA vGPU VFs require their open, reset, and close lifecycle to be
> coordinated with the PF-side nova-core driver.
>
> Add a VFIO PCI variant driver that binds NVIDIA devices only through
> driver_override and rejects non-VFs. Delegate instance lifecycle
> operations to nova-core, present the firmware-selected device and
> subsystem IDs in configuration-space reads, and adjust the reported BAR1
> aperture to the assigned profile. Use vfio-pci-core for the remaining
> VFIO operations.
>
> Signed-off-by: Zhi Wang <zhiw@nvidia.com>
> ---
> drivers/vfio/pci/Kconfig | 2 +
> drivers/vfio/pci/Makefile | 2 +
> drivers/vfio/pci/nvidia-vgpu/Kconfig | 16 ++
> drivers/vfio/pci/nvidia-vgpu/Makefile | 2 +
> drivers/vfio/pci/nvidia-vgpu/main.c | 253 ++++++++++++++++++++++++++
> 5 files changed, 275 insertions(+)
> create mode 100644 drivers/vfio/pci/nvidia-vgpu/Kconfig
> create mode 100644 drivers/vfio/pci/nvidia-vgpu/Makefile
> create mode 100644 drivers/vfio/pci/nvidia-vgpu/main.c
>
> diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig
> index 296bf01e185e..b48d8d1af42a 100644
> --- a/drivers/vfio/pci/Kconfig
> +++ b/drivers/vfio/pci/Kconfig
> @@ -74,4 +74,6 @@ source "drivers/vfio/pci/qat/Kconfig"
>
> source "drivers/vfio/pci/xe/Kconfig"
>
> +source "drivers/vfio/pci/nvidia-vgpu/Kconfig"
> +
> endmenu
> diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile
> index 6138f1bf241d..f3498e541555 100644
> --- a/drivers/vfio/pci/Makefile
> +++ b/drivers/vfio/pci/Makefile
> @@ -24,3 +24,5 @@ obj-$(CONFIG_NVGRACE_GPU_VFIO_PCI) += nvgrace-gpu/
> obj-$(CONFIG_QAT_VFIO_PCI) += qat/
>
> obj-$(CONFIG_XE_VFIO_PCI) += xe/
> +
> +obj-$(CONFIG_NVIDIA_VGPU_VFIO_PCI) += nvidia-vgpu/
> diff --git a/drivers/vfio/pci/nvidia-vgpu/Kconfig b/drivers/vfio/pci/nvidia-vgpu/Kconfig
> new file mode 100644
> index 000000000000..098822d32380
> --- /dev/null
> +++ b/drivers/vfio/pci/nvidia-vgpu/Kconfig
> @@ -0,0 +1,16 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config NVIDIA_VGPU_VFIO_PCI
> + tristate "VFIO support for the NVIDIA vGPU"
> + depends on NOVA_CORE && PCI_IOV
> + select VFIO_PCI_CORE
> + help
> + This option enables VFIO (Virtual Function I/O) support for
> + NVIDIA virtual GPUs (vGPU). It allows the assignment of a virtual
> + GPU instance to userspace applications via VFIO, typically used
> + with hypervisors such as KVM and device emulators like QEMU.
> +
> + The NVIDIA vGPU allows a physical GPU to be partitioned into
> + multiple virtual GPUs, each of which can be passed to a virtual
> + machine as a PCI device using the standard VFIO infrastructure.
> +
> + If you don't know what to do here, say N.
> diff --git a/drivers/vfio/pci/nvidia-vgpu/Makefile b/drivers/vfio/pci/nvidia-vgpu/Makefile
> new file mode 100644
> index 000000000000..193cc801a081
> --- /dev/null
> +++ b/drivers/vfio/pci/nvidia-vgpu/Makefile
> @@ -0,0 +1,2 @@
> +obj-$(CONFIG_NVIDIA_VGPU_VFIO_PCI) += nvidia-vgpu-vfio-pci.o
> +nvidia-vgpu-vfio-pci-y := main.o
> diff --git a/drivers/vfio/pci/nvidia-vgpu/main.c b/drivers/vfio/pci/nvidia-vgpu/main.c
> new file mode 100644
> index 000000000000..d8626644f952
> --- /dev/null
> +++ b/drivers/vfio/pci/nvidia-vgpu/main.c
> @@ -0,0 +1,253 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +#include <linux/module.h>
> +#include <linux/overflow.h>
> +#include <linux/pci.h>
> +#include <linux/pid.h>
> +#include <linux/vfio_pci_core.h>
> +#include <drm/nvidia_vgpu.h>
> +
> +static int nvidia_vgpu_fb_bar_index(struct pci_dev *pdev)
> +{
> + if (pci_resource_flags(pdev, 0) & IORESOURCE_MEM_64)
> + return 2;
> + return 1;
> +}
> +
> +struct nvidia_vgpu_pci_core_device {
> + struct vfio_pci_core_device core_device;
> + struct nvidia_vgpu_type_info type_info;
> + unsigned int gfid;
> +};
> +
This is more commonly called an "sbdf". Also, consider some comments.
> +static inline unsigned int nvidia_vgpu_vf_dbdf(struct pci_dev *vf)
> +{
> + return ((u32)pci_domain_nr(vf->bus) << 16) | pci_dev_id(vf);
> +}
> +
> +static int nvidia_vgpu_open_device(struct vfio_device *core_vdev)
> +{
> + struct nvidia_vgpu_pci_core_device *nvdev = container_of(
> + core_vdev, struct nvidia_vgpu_pci_core_device, core_device.vdev);
> + struct pci_dev *vf = to_pci_dev(core_vdev->dev);
> + struct nvidia_vgpu_type_info type_info;
> + int ret;
> +
> + if (!vf->is_virtfn)
> + return -ENODEV;
This is redundant to the probe check.
> +
> + ret = vfio_pci_core_enable(&nvdev->core_device);
> + if (ret)
> + return ret;
> +
> + ret = nvidia_vgpu_open(pci_physfn(vf), nvdev->gfid,
> + nvidia_vgpu_vf_dbdf(vf), task_tgid_nr(current),
> + &type_info);
> + if (ret) {
> + vfio_pci_core_disable(&nvdev->core_device);
> + return ret;
> + }
> +
> + nvdev->type_info = type_info;
> + pci_dbg(vf, "vgpu open: dev_id=0x%x subsys_id=0x%x bar1_length=0x%llx\n",
> + type_info.pci_dev_id, type_info.pci_subsys_id,
> + type_info.bar1_length);
> + vfio_pci_core_finish_enable(&nvdev->core_device);
> + return 0;
> +}
> +
> +static void nvidia_vgpu_close_device(struct vfio_device *core_vdev)
> +{
> + struct nvidia_vgpu_pci_core_device *nvdev = container_of(
> + core_vdev, struct nvidia_vgpu_pci_core_device, core_device.vdev);
> + struct pci_dev *vf = to_pci_dev(core_vdev->dev);
> +
> + nvidia_vgpu_close(pci_physfn(vf), nvdev->gfid);
> + vfio_pci_core_close_device(core_vdev);
> +}
> +
> +static ssize_t nvidia_vgpu_pci_read_config(struct vfio_device *core_vdev,
> + char __user *buf, size_t count,
> + loff_t *ppos)
> +{
> + struct nvidia_vgpu_pci_core_device *nvdev = container_of(
> + core_vdev, struct nvidia_vgpu_pci_core_device, core_device.vdev);
> + struct nvidia_vgpu_type_info *ti = &nvdev->type_info;
> + loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK;
> + size_t register_offset;
> + loff_t copy_offset;
> + size_t copy_count;
> + __le16 val16;
> + int ret;
> +
> + ret = vfio_pci_core_read(core_vdev, buf, count, ppos);
> + if (ret < 0)
> + return ret;
> +
> + if (vfio_pci_core_range_intersect_range(pos, count, PCI_DEVICE_ID,
> + sizeof(val16), ©_offset,
> + ©_count, ®ister_offset)) {
> + val16 = cpu_to_le16(ti->pci_dev_id);
> + if (copy_to_user(buf + copy_offset,
> + (void *)&val16 + register_offset, copy_count))
> + return -EFAULT;
> + }
Just stuff the device ID into vconfig, it's already read from there.
> +
> + if (vfio_pci_core_range_intersect_range(pos, count, PCI_SUBSYSTEM_ID,
> + sizeof(val16), ©_offset,
> + ©_count, ®ister_offset)) {
> + val16 = cpu_to_le16(ti->pci_subsys_id);
> + if (copy_to_user(buf + copy_offset,
> + (void *)&val16 + register_offset, copy_count))
> + return -EFAULT;
> + }
There's possibly an argument to be made that subsystem ID could be read
from vconfig by default too so it could be pre-filled after
vfio_config_init(), ie. after vfio_pci_core_enable(). The only reason
it might change would be if firmware was updated, but a firmware update
through vfio that changes the subsystem ID would be pretty sketchy
already.
> +
> + return count;
> +}
> +
> +static ssize_t nvidia_vgpu_pci_read(struct vfio_device *core_vdev,
> + char __user *buf, size_t count,
> + loff_t *ppos)
> +{
> + unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
> +
> + if (index == VFIO_PCI_CONFIG_REGION_INDEX)
> + return nvidia_vgpu_pci_read_config(core_vdev, buf, count, ppos);
> +
> + return vfio_pci_core_read(core_vdev, buf, count, ppos);
> +}
> +
> +static int nvidia_vgpu_bar1_size(struct nvidia_vgpu_pci_core_device *nvdev,
> + u64 *size)
> +{
> + if (check_shl_overflow(nvdev->type_info.bar1_length, 20, size))
> + return -EOVERFLOW;
> +
> + return 0;
> +}
> +
> +static int nvidia_vgpu_get_region_info(struct vfio_device *core_vdev,
> + struct vfio_region_info *info,
> + struct vfio_info_cap *caps)
> +{
> + int ret;
> +
> + ret = vfio_pci_ioctl_get_region_info(core_vdev, info, caps);
> + if (ret)
> + return ret;
> +
> + if (info->index == nvidia_vgpu_fb_bar_index(
> + to_pci_dev(core_vdev->dev)) && info->size) {
> + struct nvidia_vgpu_pci_core_device *nvdev = container_of(
> + core_vdev, struct nvidia_vgpu_pci_core_device,
> + core_device.vdev);
> + u64 vgpu_bar1;
> +
> + ret = nvidia_vgpu_bar1_size(nvdev, &vgpu_bar1);
> + if (ret)
> + return ret;
> +
> + if (vgpu_bar1 && vgpu_bar1 < info->size)
> + info->size = vgpu_bar1;
> + }
So we're changing the reported BAR1 size, but just trusting that
userspace honors that size for read/write/mmap? Again, consider some
comments.
> +
> + return 0;
> +}
> +
> +static long nvidia_vgpu_pci_ioctl(struct vfio_device *core_vdev,
> + unsigned int cmd, unsigned long arg)
> +{
> + if (cmd == VFIO_DEVICE_RESET) {
> + struct nvidia_vgpu_pci_core_device *nvdev = container_of(
> + core_vdev, struct nvidia_vgpu_pci_core_device,
> + core_device.vdev);
> + struct pci_dev *vf = to_pci_dev(core_vdev->dev);
> + int ret;
> +
> + ret = nvidia_vgpu_reset(pci_physfn(vf), nvdev->gfid);
> + if (ret)
> + return ret;
> + }
> +
> + return vfio_pci_core_ioctl(core_vdev, cmd, arg);
What about reset invoked through FLR? Would this be better served
through .reset_prepare and .reset_done?
> +}
> +
> +static const struct vfio_device_ops nvidia_vgpu_pci_ops = {
> + .name = "nvidia-vgpu-vfio-pci",
> + .init = vfio_pci_core_init_dev,
> + .release = vfio_pci_core_release_dev,
> + .open_device = nvidia_vgpu_open_device,
> + .close_device = nvidia_vgpu_close_device,
> + .ioctl = nvidia_vgpu_pci_ioctl,
> + .get_region_info_caps = nvidia_vgpu_get_region_info,
> + .device_feature = vfio_pci_core_ioctl_feature,
> + .read = nvidia_vgpu_pci_read,
> + .write = vfio_pci_core_write,
> + .mmap = vfio_pci_core_mmap,
> + .request = vfio_pci_core_request,
> + .match = vfio_pci_core_match,
> + .match_token_uuid = vfio_pci_core_match_token_uuid,
> + .bind_iommufd = vfio_iommufd_physical_bind,
> + .unbind_iommufd = vfio_iommufd_physical_unbind,
> + .attach_ioas = vfio_iommufd_physical_attach_ioas,
> + .detach_ioas = vfio_iommufd_physical_detach_ioas,
> +};
> +
> +static int nvidia_vgpu_pci_probe(struct pci_dev *pdev,
> + const struct pci_device_id *id)
> +{
> + struct nvidia_vgpu_pci_core_device *nvdev;
> + int vf_id;
> + int ret;
> +
> + if (!pdev->is_virtfn)
> + return -ENODEV;
This driver needs to bind to what it matches in the id table, we don't
have a policy for userspace to pick a 2nd best variant driver.
hisi_acc handles a similar situation where only the VFs are supported
for the migration feature of the variant driver. The PF needs to be
supported here and bind to a vfio-pci-core passthrough ops structure.
We should probably define a PCI_DRIVER_OVERRIDE_DEVICE_VFIO variant
that allows a class code to be specified so we aren't using this for
all 10de: devices. I'm hoping that class code is for a 3D accelerator
or the like rather than VGA class (a VF can't technically support a
legacy endpoint anyway), but we need to consider what existing devices
that currently use vfio-pci would now be bound to this driver and what
module option features they might use. Thanks,
Alex
> +
> + vf_id = pci_iov_vf_id(pdev);
> + if (vf_id < 0)
> + return vf_id;
> +
> + nvdev = vfio_alloc_device(nvidia_vgpu_pci_core_device, core_device.vdev,
> + &pdev->dev, &nvidia_vgpu_pci_ops);
> + if (IS_ERR(nvdev))
> + return PTR_ERR(nvdev);
> +
> + nvdev->gfid = vf_id + 1;
> + dev_set_drvdata(&pdev->dev, &nvdev->core_device);
> + ret = vfio_pci_core_register_device(&nvdev->core_device);
> + if (ret)
> + goto out_put_vdev;
> +
> + return 0;
> +
> +out_put_vdev:
> + vfio_put_device(&nvdev->core_device.vdev);
> + return ret;
> +}
> +
> +static void nvidia_vgpu_pci_remove(struct pci_dev *pdev)
> +{
> + struct vfio_pci_core_device *core_device = dev_get_drvdata(&pdev->dev);
> +
> + vfio_pci_core_unregister_device(core_device);
> + vfio_put_device(&core_device->vdev);
> +}
> +
> +static const struct pci_device_id nvidia_vgpu_pci_table[] = {
> + /* Placeholder: match all NVIDIA VFs (vendor 0x10de) */
> + { PCI_DRIVER_OVERRIDE_DEVICE_VFIO(PCI_VENDOR_ID_NVIDIA, PCI_ANY_ID) },
> + {}
> +};
> +MODULE_DEVICE_TABLE(pci, nvidia_vgpu_pci_table);
> +
> +static struct pci_driver nvidia_vgpu_pci_driver = {
> + .name = "nvidia-vgpu-vfio-pci",
> + .id_table = nvidia_vgpu_pci_table,
> + .probe = nvidia_vgpu_pci_probe,
> + .remove = nvidia_vgpu_pci_remove,
> + .driver_managed_dma = true,
> +};
> +module_pci_driver(nvidia_vgpu_pci_driver);
> +
> +MODULE_DESCRIPTION("NVIDIA vGPU vfio-pci driver");
> +MODULE_LICENSE("GPL");
> +MODULE_IMPORT_NS("NOVA_CORE_VGPU");
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2026-09-09 3:00 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-05 8:11 [PATCH 00/13] Introduce NVIDIA vGPU manager and VFIO variant driver Zhi Wang
2026-09-05 8:11 ` [PATCH 01/13] gpu: nova-core: vgpu: add post-GSP-boot vGPU initialization Zhi Wang
2026-09-05 8:11 ` [PATCH 02/13] gpu: nova-core: mm: add VramBlock and Bar1Map Zhi Wang
2026-09-05 8:11 ` [PATCH 03/13] gpu: nova-core: vgpu: add VRAM slot allocator Zhi Wang
2026-09-05 8:11 ` [PATCH 04/13] gpu: nova-core: vgpu: add r000 plugin bindings Zhi Wang
2026-09-05 8:11 ` [PATCH 05/13] gpu: nova-core: vgpu: add instance create/destroy Zhi Wang
2026-09-05 8:11 ` [PATCH 06/13] gpu: nova-core: gsp: add GMC transaction helpers Zhi Wang
2026-09-05 8:11 ` [PATCH 07/13] gpu: nova-core: vgpu: add vGPU bootload Zhi Wang
2026-09-05 8:11 ` [PATCH 08/13] gpu: nova-core: vgpu: implement PluginRpc channel and config params Zhi Wang
2026-09-05 8:11 ` [PATCH 09/13] gpu: nova-core: vgpu: scrub guest framebuffer memory with CeUtils Zhi Wang
2026-09-05 8:11 ` [PATCH 10/13] gpu: nova-core: vgpu: export plugin log buffers via debugfs Zhi Wang
2026-09-05 8:11 ` [PATCH 11/13] gpu: nova-core: vgpu: export lifecycle operations to VFIO Zhi Wang
2026-09-05 8:11 ` [PATCH 12/13] vfio/nvidia-vgpu: add the NVIDIA vGPU VFIO variant driver Zhi Wang
2026-09-09 3:00 ` Alex Williamson
2026-09-05 8:11 ` [PATCH 13/13] gpu: nova-core: reserve the 48-VM WPR2 heap Zhi Wang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox