[PATCH 0/2] gpu: nova-core: add fwctl driver

[PATCH 0/2] gpu: nova-core: add fwctl driver

[Zhi Wang]

This series adds a fwctl driver to nova-core, enabling userspace to issue
firmware commands to the GSP through the standard fwctl ioctl
interface. E.g. uploading the vGPU type blobs and managing the vGPU
lifecycle, which is previously implemented in sysfs.

Patch 1 fixes a boot-order issue: since Rust fwctl abstractions force
fwctl to be built-in (Kconfig bool), the fwctl class must be registered
before PCI drivers probe. Switching from module_init to subsys_initcall
ensures this.

Patch 2 adds the nova-core fwctl driver itself. It defines a UAPI for
nova-core specific RPC commands and wires them through to the GSP via
the existing RM control command infrastructure. As a first user, the
UPLOAD_VGPU_TYPE command is added, which uploads vGPU type definitions
to the GSP. It is a required step before any vGPU can be created.

This patchset depends on several series:

- [PATCH v5 0/9] gpu: nova-core: gsp: add continuation record support [1]
- [PATCH v3 0/5] gpu: nova-core: gsp: add locking to Cmdq [2]
- [PATCH v3 0/1] rust: introduce abstractions for fwctl [3]

And a tree with all the patches can be found here.[4]

[1] https://lore.kernel.org/nouveau/20260304-cmdq-continuation-v5-0-3f19d759ed93@nvidia.com/T/#t
[2] https://lore.kernel.org/nouveau/DGTZATU21WV2.1YRBLUAXFHMVJ@nvidia.com/T/#t
[3] https://lore.kernel.org/all/20260217204909.211793-1-zhiw@nvidia.com/
[4] https://github.com/zhiwang-nvidia/nova-core/tree/nova-core/fwctl-driver-v1

Zhi Wang (2):
  fwctl: use subsys_initcall for built-in configuration
  gpu: nova-core: add fwctl driver for firmware control interface

 drivers/fwctl/main.c                          |  2 +-
 drivers/gpu/nova-core/fwctl.rs                | 99 +++++++++++++++++++
 drivers/gpu/nova-core/gpu.rs                  |  4 +-
 drivers/gpu/nova-core/gsp.rs                  | 25 ++++-
 drivers/gpu/nova-core/gsp/boot.rs             |  6 +-
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs |  1 +
 drivers/gpu/nova-core/gsp/fw/rm.rs            |  5 +
 drivers/gpu/nova-core/gsp/rm/commands.rs      |  4 +-
 drivers/gpu/nova-core/nova_core.rs            |  2 +
 include/uapi/fwctl/fwctl.h                    |  1 +
 include/uapi/fwctl/nova-core.h                | 52 ++++++++++
 rust/kernel/fwctl.rs                          |  2 +
 rust/uapi/uapi_helper.h                       |  1 +
 13 files changed, 194 insertions(+), 10 deletions(-)
 create mode 100644 drivers/gpu/nova-core/fwctl.rs
 create mode 100644 include/uapi/fwctl/nova-core.h

-- 
2.51.0

[PATCH 1/2] fwctl: use subsys_initcall for built-in configuration

[Zhi Wang]

Rust does not yet support loadable kernel modules, so
RUST_FWCTL_ABSTRACTIONS is a Kconfig bool, which forces fwctl to be
built-in. With module_init, fwctl initializes too late and PCI drivers
probe before the fwctl class is registered.

Switch to subsys_initcall.

Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
 drivers/fwctl/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/fwctl/main.c b/drivers/fwctl/main.c
index bc6378506296..098c3824ad75 100644
--- a/drivers/fwctl/main.c
+++ b/drivers/fwctl/main.c
@@ -415,7 +415,7 @@ static void __exit fwctl_exit(void)
 	unregister_chrdev_region(fwctl_dev, FWCTL_MAX_DEVICES);
 }
 
-module_init(fwctl_init);
+subsys_initcall(fwctl_init);
 module_exit(fwctl_exit);
 MODULE_DESCRIPTION("fwctl device firmware access framework");
 MODULE_LICENSE("GPL");
-- 
2.51.0

[PATCH 2/2] gpu: nova-core: add fwctl driver for firmware control interface

[Zhi Wang]

Add a fwctl driver that provides an infrastructure for userspace to
issue firmware commands to the GSP. New commands can be added by
extending RmControlMsgFunction and the fwctl command dispatch table.

As a first user, add FWCTL_CMD_NOVA_CORE_UPLOAD_VGPU_TYPE, which uploads
vGPU type definitions to the GSP. It is a required step before any vGPU
can be created.

Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
 drivers/gpu/nova-core/fwctl.rs                | 99 +++++++++++++++++++
 drivers/gpu/nova-core/gpu.rs                  |  4 +-
 drivers/gpu/nova-core/gsp.rs                  | 25 ++++-
 drivers/gpu/nova-core/gsp/boot.rs             |  6 +-
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs |  1 +
 drivers/gpu/nova-core/gsp/fw/rm.rs            |  5 +
 drivers/gpu/nova-core/gsp/rm/commands.rs      |  4 +-
 drivers/gpu/nova-core/nova_core.rs            |  2 +
 include/uapi/fwctl/fwctl.h                    |  1 +
 include/uapi/fwctl/nova-core.h                | 52 ++++++++++
 rust/kernel/fwctl.rs                          |  2 +
 rust/uapi/uapi_helper.h                       |  1 +
 12 files changed, 193 insertions(+), 9 deletions(-)
 create mode 100644 drivers/gpu/nova-core/fwctl.rs
 create mode 100644 include/uapi/fwctl/nova-core.h

diff --git a/drivers/gpu/nova-core/fwctl.rs b/drivers/gpu/nova-core/fwctl.rs
new file mode 100644
index 000000000000..9ec147a815db
--- /dev/null
+++ b/drivers/gpu/nova-core/fwctl.rs
@@ -0,0 +1,99 @@
+// SPDX-License-Identifier: GPL-2.0
+
+use kernel::{
+    fwctl::{
+        self,
+        DeviceType,
+        FwRpcResponse,
+        Operations,
+        RpcScope, //
+    },
+    prelude::*,
+    transmute::{AsBytes, FromBytes},
+    uapi, //
+};
+
+use crate::{
+    driver::NovaCore,
+    gsp::{
+        RmControlMsgFunction,
+        rm::commands::send_rm_control, //
+    },
+};
+
+/// Byte-serializable wrapper for [`uapi::fwctl_rpc_nova_core_request_hdr`].
+#[repr(transparent)]
+struct FwctlNovaCoreReqHdr(uapi::fwctl_rpc_nova_core_request_hdr);
+
+// SAFETY: All fields are plain `__u32` with no padding.
+unsafe impl FromBytes for FwctlNovaCoreReqHdr {}
+
+/// Byte-serializable wrapper for [`uapi::fwctl_rpc_nova_core_resp_hdr`].
+#[repr(transparent)]
+struct FwctlNovaCoreRespHdr(uapi::fwctl_rpc_nova_core_resp_hdr);
+
+// SAFETY: All fields are plain `__u32` with no padding.
+unsafe impl AsBytes for FwctlNovaCoreRespHdr {}
+
+/// Per-FD fwctl user context and operations for nova-core.
+pub(crate) struct NovaCoreFwCtl;
+
+impl Operations for NovaCoreFwCtl {
+    type DeviceData = ();
+
+    const DEVICE_TYPE: DeviceType = DeviceType::NovaCore;
+
+    fn open(_device: &fwctl::Device<Self>) -> Result<impl PinInit<Self, Error>, Error> {
+        Ok(Ok(NovaCoreFwCtl))
+    }
+
+    fn fw_rpc(
+        _this: &Self,
+        device: &fwctl::Device<Self>,
+        scope: RpcScope,
+        rpc_in: &mut [u8],
+    ) -> Result<FwRpcResponse, Error> {
+        let hdr_size = size_of::<FwctlNovaCoreReqHdr>();
+
+        if rpc_in.len() < hdr_size {
+            return Err(EINVAL);
+        }
+
+        if scope != RpcScope::Configuration {
+            return Err(EPERM);
+        }
+
+        let (hdr, _) = FwctlNovaCoreReqHdr::from_bytes_prefix(rpc_in).ok_or(EINVAL)?;
+        let cmd = hdr.0.cmd;
+
+        let rm_cmd = match cmd {
+            uapi::fwctl_cmd_nova_core_FWCTL_CMD_NOVA_CORE_UPLOAD_VGPU_TYPE => {
+                RmControlMsgFunction::VgpuMgrInternalPgpuAddVgpuType
+            }
+            _ => return Err(EINVAL),
+        };
+
+        let parent = device.parent();
+        let data = parent.drvdata::<NovaCore>()?;
+        let bar = data.gpu.bar.as_ref().access(parent)?;
+
+        let params = &rpc_in[hdr_size..];
+        let reply_params = send_rm_control(
+            &data.gpu.gsp.cmdq,
+            bar,
+            data.gpu.gsp.h_client,
+            data.gpu.gsp.h_subdevice,
+            rm_cmd,
+            params,
+        )?;
+
+        let resp_hdr = FwctlNovaCoreRespHdr(uapi::fwctl_rpc_nova_core_resp_hdr {
+            mctp_header: 0,
+            nvdm_header: 0,
+        });
+        let mut out = KVec::new();
+        out.extend_from_slice(resp_hdr.as_bytes(), GFP_KERNEL)?;
+        out.extend_from_slice(&reply_params, GFP_KERNEL)?;
+        Ok(FwRpcResponse::NewBuffer(out))
+    }
+}
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index 60c85fffaeaf..7965ce37eb08 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -241,7 +241,7 @@ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
 pub(crate) struct Gpu {
     spec: Spec,
     /// MMIO mapping of PCI BAR 0
-    bar: Arc<Devres<Bar0>>,
+    pub(crate) bar: Arc<Devres<Bar0>>,
     /// System memory page required for flushing all pending GPU-side memory writes done through
     /// PCIE into system memory, via sysmembar (A GPU-initiated HW memory-barrier operation).
     sysmem_flush: SysmemFlush,
@@ -251,7 +251,7 @@ pub(crate) struct Gpu {
     sec2_falcon: Falcon<Sec2Falcon>,
     /// GSP runtime data. Temporarily an empty placeholder.
     #[pin]
-    gsp: Gsp,
+    pub(crate) gsp: Gsp,
 }
 
 impl Gpu {
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index 1a1c4e9808ac..77eb30010c2f 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -4,11 +4,13 @@
 
 use kernel::{
     device,
+    devres::Devres,
     dma::{
         CoherentAllocation,
         DmaAddress, //
     },
     dma_write,
+    fwctl,
     pci,
     prelude::*,
     transmute::AsBytes, //
@@ -21,15 +23,19 @@
 mod sequencer;
 
 pub(crate) use fw::{
+    rm::RmControlMsgFunction,
     GspFwWprMeta,
     LibosParams, //
 };
 
 use crate::{
-    gsp::cmdq::Cmdq,
-    gsp::fw::{
-        GspArgumentsPadded,
-        LibosMemoryRegionInitArgument, //
+    fwctl::NovaCoreFwCtl,
+    gsp::{
+        cmdq::Cmdq,
+        fw::{
+            GspArgumentsPadded,
+            LibosMemoryRegionInitArgument, //
+        },
     },
     num,
 };
@@ -117,6 +123,12 @@ pub(crate) struct Gsp {
     pub(crate) cmdq: Cmdq,
     /// RM arguments.
     rmargs: CoherentAllocation<GspArgumentsPadded>,
+    /// Cached RM internal client handle from GSP static info.
+    pub(crate) h_client: u32,
+    /// Cached RM internal subdevice handle from GSP static info.
+    pub(crate) h_subdevice: u32,
+    /// fwctl registration for userspace RM control.
+    fwctl: Pin<KBox<Devres<fwctl::Registration<NovaCoreFwCtl>>>>,
 }
 
 impl Gsp {
@@ -125,6 +137,8 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> impl PinInit<Self, Error
         pin_init::pin_init_scope(move || {
             let dev = pdev.as_ref();
 
+            let fwctl_dev = fwctl::Device::<NovaCoreFwCtl>::new(pdev.as_ref(), Ok(()))?;
+
             Ok(try_pin_init!(Self {
                 libos: CoherentAllocation::<LibosMemoryRegionInitArgument>::alloc_coherent(
                     dev,
@@ -140,6 +154,9 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> impl PinInit<Self, Error
                     1,
                     GFP_KERNEL | __GFP_ZERO,
                 )?,
+                h_client: 0,
+                h_subdevice: 0,
+                fwctl: KBox::pin_init(fwctl::Registration::new(pdev.as_ref(), &fwctl_dev), GFP_KERNEL)?,
                 _: {
                     // Initialise the logging structures. The OpenRM equivalents are in:
                     // _kgspInitLibosLoggingStructures (allocates memory for buffers)
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index bc53e667cd9e..f493546b78ff 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -128,7 +128,7 @@ fn run_fwsec_frts(
     ///
     /// Upon return, the GSP is up and running, and its runtime object given as return value.
     pub(crate) fn boot(
-        self: Pin<&mut Self>,
+        mut self: Pin<&mut Self>,
         pdev: &pci::Device<device::Bound>,
         bar: &Bar0,
         chipset: Chipset,
@@ -221,6 +221,10 @@ pub(crate) fn boot(
 
         // Obtain and display basic GPU information.
         let info = commands::get_gsp_info(&self.cmdq, bar)?;
+        // SAFETY: h_client and h_subdevice are not structurally pinned.
+        let this = unsafe { self.as_mut().get_unchecked_mut() };
+        this.h_client = info.h_client();
+        this.h_subdevice = info.h_subdevice();
         match info.gpu_name() {
             Ok(name) => dev_info!(pdev, "GPU name: {}\n", name),
             Err(e) => dev_warn!(pdev, "GPU name unavailable: {:?}\n", e),
diff --git a/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs b/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs
index 354ee2cfa295..b5b16451a507 100644
--- a/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs
+++ b/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs
@@ -45,6 +45,7 @@ fn fmt(&self, fmt: &mut ::core::fmt::Formatter<'_>) -> ::core::fmt::Result {
 pub const REGISTRY_TABLE_ENTRY_TYPE_DWORD: u32 = 1;
 pub const GSP_MSG_QUEUE_ELEMENT_SIZE_MAX: u32 = 65536;
 pub const NV2080_CTRL_CMD_CE_GET_FAULT_METHOD_BUFFER_SIZE: u32 = 545270280;
+pub const NV2080_CTRL_CMD_VGPU_MGR_INTERNAL_PGPU_ADD_VGPU_TYPE: u32 = 545275907;
 pub type __u8 = ffi::c_uchar;
 pub type __u16 = ffi::c_ushort;
 pub type __u32 = ffi::c_uint;
diff --git a/drivers/gpu/nova-core/gsp/fw/rm.rs b/drivers/gpu/nova-core/gsp/fw/rm.rs
index 9f1e3546d39d..c0b6e90ee135 100644
--- a/drivers/gpu/nova-core/gsp/fw/rm.rs
+++ b/drivers/gpu/nova-core/gsp/fw/rm.rs
@@ -19,6 +19,8 @@
 pub(crate) enum RmControlMsgFunction {
     /// Get the CE fault method buffer size.
     CeGetFaultMethodBufferSize = bindings::NV2080_CTRL_CMD_CE_GET_FAULT_METHOD_BUFFER_SIZE,
+    /// Upload vGPU type definitions to GSP.
+    VgpuMgrInternalPgpuAddVgpuType = bindings::NV2080_CTRL_CMD_VGPU_MGR_INTERNAL_PGPU_ADD_VGPU_TYPE,
 }
 
 impl TryFrom<u32> for RmControlMsgFunction {
@@ -29,6 +31,9 @@ fn try_from(value: u32) -> Result<Self> {
             bindings::NV2080_CTRL_CMD_CE_GET_FAULT_METHOD_BUFFER_SIZE => {
                 Ok(Self::CeGetFaultMethodBufferSize)
             }
+            bindings::NV2080_CTRL_CMD_VGPU_MGR_INTERNAL_PGPU_ADD_VGPU_TYPE => {
+                Ok(Self::VgpuMgrInternalPgpuAddVgpuType)
+            }
             _ => Err(EINVAL),
         }
     }
diff --git a/drivers/gpu/nova-core/gsp/rm/commands.rs b/drivers/gpu/nova-core/gsp/rm/commands.rs
index 1d045e6f1afb..02670899d2bc 100644
--- a/drivers/gpu/nova-core/gsp/rm/commands.rs
+++ b/drivers/gpu/nova-core/gsp/rm/commands.rs
@@ -98,7 +98,7 @@ fn read(
 }
 
 /// Sends an RM control command, checks the reply status, and returns the raw parameter bytes.
-fn send_rm_control(
+pub(crate) fn send_rm_control(
     cmdq: &Cmdq,
     bar: &Bar0,
     h_client: u32,
@@ -106,7 +106,7 @@ fn send_rm_control(
     cmd: RmControlMsgFunction,
     params: &[u8],
 ) -> Result<KVVec<u8>> {
-    let reply = cmdq.send_sync_command(bar, RmControl::new(h_client, h_object, cmd, params))?;
+    let reply = cmdq.send_command(bar, RmControl::new(h_client, h_object, cmd, params))?;
 
     Result::from(reply.status)?;
 
diff --git a/drivers/gpu/nova-core/nova_core.rs b/drivers/gpu/nova-core/nova_core.rs
index b5caf1044697..863dc041272c 100644
--- a/drivers/gpu/nova-core/nova_core.rs
+++ b/drivers/gpu/nova-core/nova_core.rs
@@ -10,6 +10,7 @@
 mod falcon;
 mod fb;
 mod firmware;
+mod fwctl;
 mod gfw;
 mod gpu;
 mod gsp;
@@ -27,6 +28,7 @@
     description: "Nova Core GPU driver",
     license: "GPL v2",
     firmware: [],
+    imports_ns: ["FWCTL"],
 }
 
 kernel::module_firmware!(firmware::ModInfoBuilder);
diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h
index 716ac0eee42d..f6289fbf3062 100644
--- a/include/uapi/fwctl/fwctl.h
+++ b/include/uapi/fwctl/fwctl.h
@@ -45,6 +45,7 @@ enum fwctl_device_type {
 	FWCTL_DEVICE_TYPE_MLX5 = 1,
 	FWCTL_DEVICE_TYPE_CXL = 2,
 	FWCTL_DEVICE_TYPE_PDS = 4,
+	FWCTL_DEVICE_TYPE_NOVA_CORE = 5,
 };
 
 /**
diff --git a/include/uapi/fwctl/nova-core.h b/include/uapi/fwctl/nova-core.h
new file mode 100644
index 000000000000..3f1d94b44ec8
--- /dev/null
+++ b/include/uapi/fwctl/nova-core.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES
+ *
+ * nova-core fwctl device specific definitions.
+ *
+ * The device_type for this file is FWCTL_DEVICE_TYPE_NOVA_CORE.
+ */
+#ifndef _UAPI_FWCTL_NOVA_CORE_H
+#define _UAPI_FWCTL_NOVA_CORE_H
+
+#include <linux/types.h>
+
+/**
+ * enum fwctl_cmd_nova_core - Firmware command identifiers
+ * @FWCTL_CMD_NOVA_CORE_UPLOAD_VGPU_TYPE: Upload vGPU type definitions to GSP.
+ *     Payload is NV2080_CTRL_VGPU_MGR_INTERNAL_PGPU_ADD_VGPU_TYPE_PARAMS.
+ */
+enum fwctl_cmd_nova_core {
+	FWCTL_CMD_NOVA_CORE_UPLOAD_VGPU_TYPE = 0,
+};
+
+/**
+ * struct fwctl_rpc_nova_core_request_hdr - ioctl(FWCTL_RPC) input header
+ * @cmd: Command identifier from &enum fwctl_cmd_nova_core.
+ * @mctp_header: MCTP transport header (packed u32).
+ * @nvdm_header: NVDM vendor-defined message header (packed u32).
+ *
+ * Placed at &struct fwctl_rpc.in with total length &struct fwctl_rpc.in_len.
+ * The access scope is specified through &struct fwctl_rpc.scope.
+ * Followed by command-specific input parameters.
+ */
+struct fwctl_rpc_nova_core_request_hdr {
+	__u32 mctp_header;
+	__u32 nvdm_header;
+	__u32 cmd;
+};
+
+/**
+ * struct fwctl_rpc_nova_core_resp_hdr - ioctl(FWCTL_RPC) output header
+ * @mctp_header: MCTP transport header (packed u32).
+ * @nvdm_header: NVDM vendor-defined message header (packed u32).
+ *
+ * Placed at &struct fwctl_rpc.out with total length &struct fwctl_rpc.out_len.
+ * Followed by command-specific output parameters.
+ */
+struct fwctl_rpc_nova_core_resp_hdr {
+	__u32 mctp_header;
+	__u32 nvdm_header;
+};
+
+#endif
diff --git a/rust/kernel/fwctl.rs b/rust/kernel/fwctl.rs
index bbced4761e0c..345e96144028 100644
--- a/rust/kernel/fwctl.rs
+++ b/rust/kernel/fwctl.rs
@@ -34,6 +34,8 @@ pub enum DeviceType {
     Cxl = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_CXL,
     /// AMD/Pensando PDS device.
     Pds = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_PDS,
+    /// NVIDIA Nova Core GPU device.
+    NovaCore = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_NOVA_CORE,
 }
 
 impl From<DeviceType> for u32 {
diff --git a/rust/uapi/uapi_helper.h b/rust/uapi/uapi_helper.h
index 06d7d1a2e8da..df3309afc055 100644
--- a/rust/uapi/uapi_helper.h
+++ b/rust/uapi/uapi_helper.h
@@ -10,6 +10,7 @@
 #include <uapi/drm/drm.h>
 #include <uapi/drm/nova_drm.h>
 #include <uapi/drm/panthor_drm.h>
+#include <uapi/fwctl/nova-core.h>
 #include <uapi/linux/android/binder.h>
 #include <uapi/linux/mdio.h>
 #include <uapi/linux/mii.h>
-- 
2.51.0

Re: [PATCH 2/2] gpu: nova-core: add fwctl driver for firmware control interface

[Danilo Krummrich]

On Thu Mar 5, 2026 at 8:09 PM CET, Zhi Wang wrote:
> +use kernel::{
> +    fwctl::{
> +        self,
> +        DeviceType,
> +        FwRpcResponse,
> +        Operations,
> +        RpcScope, //
> +    },
> +    prelude::*,
> +    transmute::{AsBytes, FromBytes},

NIT: Formatting.

> +    uapi, //
> +};
> +
> +use crate::{
> +    driver::NovaCore,
> +    gsp::{
> +        RmControlMsgFunction,
> +        rm::commands::send_rm_control, //
> +    },
> +};
> +
> +/// Byte-serializable wrapper for [`uapi::fwctl_rpc_nova_core_request_hdr`].
> +#[repr(transparent)]
> +struct FwctlNovaCoreReqHdr(uapi::fwctl_rpc_nova_core_request_hdr);
> +
> +// SAFETY: All fields are plain `__u32` with no padding.
> +unsafe impl FromBytes for FwctlNovaCoreReqHdr {}
> +
> +/// Byte-serializable wrapper for [`uapi::fwctl_rpc_nova_core_resp_hdr`].
> +#[repr(transparent)]
> +struct FwctlNovaCoreRespHdr(uapi::fwctl_rpc_nova_core_resp_hdr);
> +
> +// SAFETY: All fields are plain `__u32` with no padding.
> +unsafe impl AsBytes for FwctlNovaCoreRespHdr {}
> +
> +/// Per-FD fwctl user context and operations for nova-core.
> +pub(crate) struct NovaCoreFwCtl;
> +
> +impl Operations for NovaCoreFwCtl {
> +    type DeviceData = ();
> +
> +    const DEVICE_TYPE: DeviceType = DeviceType::NovaCore;
> +
> +    fn open(_device: &fwctl::Device<Self>) -> Result<impl PinInit<Self, Error>, Error> {
> +        Ok(Ok(NovaCoreFwCtl))
> +    }
> +
> +    fn fw_rpc(
> +        _this: &Self,
> +        device: &fwctl::Device<Self>,
> +        scope: RpcScope,
> +        rpc_in: &mut [u8],
> +    ) -> Result<FwRpcResponse, Error> {

Result<FwRpcResponse>

> +        let hdr_size = size_of::<FwctlNovaCoreReqHdr>();
> +
> +        if rpc_in.len() < hdr_size {
> +            return Err(EINVAL);
> +        }
> +
> +        if scope != RpcScope::Configuration {
> +            return Err(EPERM);
> +        }
> +
> +        let (hdr, _) = FwctlNovaCoreReqHdr::from_bytes_prefix(rpc_in).ok_or(EINVAL)?;
> +        let cmd = hdr.0.cmd;
> +
> +        let rm_cmd = match cmd {
> +            uapi::fwctl_cmd_nova_core_FWCTL_CMD_NOVA_CORE_UPLOAD_VGPU_TYPE => {
> +                RmControlMsgFunction::VgpuMgrInternalPgpuAddVgpuType
> +            }
> +            _ => return Err(EINVAL),
> +        };
> +
> +        let parent = device.parent();
> +        let data = parent.drvdata::<NovaCore>()?;

Please don't use drvdata() for this, it is not what it is intended for and I
kinda regret a bit that I added it in the first place.

Instead, please use your fwctl::Device private data, i.e. Self.

> +        let bar = data.gpu.bar.as_ref().access(parent)?;
> +
> +        let params = &rpc_in[hdr_size..];
> +        let reply_params = send_rm_control(
> +            &data.gpu.gsp.cmdq,
> +            bar,
> +            data.gpu.gsp.h_client,
> +            data.gpu.gsp.h_subdevice,
> +            rm_cmd,
> +            params,
> +        )?;
> +
> +        let resp_hdr = FwctlNovaCoreRespHdr(uapi::fwctl_rpc_nova_core_resp_hdr {
> +            mctp_header: 0,
> +            nvdm_header: 0,
> +        });
> +        let mut out = KVec::new();
> +        out.extend_from_slice(resp_hdr.as_bytes(), GFP_KERNEL)?;
> +        out.extend_from_slice(&reply_params, GFP_KERNEL)?;
> +        Ok(FwRpcResponse::NewBuffer(out))
> +    }
> +}
> diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
> index 60c85fffaeaf..7965ce37eb08 100644
> --- a/drivers/gpu/nova-core/gpu.rs
> +++ b/drivers/gpu/nova-core/gpu.rs
> @@ -241,7 +241,7 @@ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
>  pub(crate) struct Gpu {
>      spec: Spec,
>      /// MMIO mapping of PCI BAR 0
> -    bar: Arc<Devres<Bar0>>,
> +    pub(crate) bar: Arc<Devres<Bar0>>,
>      /// System memory page required for flushing all pending GPU-side memory writes done through
>      /// PCIE into system memory, via sysmembar (A GPU-initiated HW memory-barrier operation).
>      sysmem_flush: SysmemFlush,
> @@ -251,7 +251,7 @@ pub(crate) struct Gpu {
>      sec2_falcon: Falcon<Sec2Falcon>,
>      /// GSP runtime data. Temporarily an empty placeholder.
>      #[pin]
> -    gsp: Gsp,
> +    pub(crate) gsp: Gsp,
>  }
>  
>  impl Gpu {
> diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
> index 1a1c4e9808ac..77eb30010c2f 100644
> --- a/drivers/gpu/nova-core/gsp.rs
> +++ b/drivers/gpu/nova-core/gsp.rs
> @@ -4,11 +4,13 @@
>  
>  use kernel::{
>      device,
> +    devres::Devres,
>      dma::{
>          CoherentAllocation,
>          DmaAddress, //
>      },
>      dma_write,
> +    fwctl,
>      pci,
>      prelude::*,
>      transmute::AsBytes, //
> @@ -21,15 +23,19 @@
>  mod sequencer;
>  
>  pub(crate) use fw::{
> +    rm::RmControlMsgFunction,
>      GspFwWprMeta,
>      LibosParams, //
>  };
>  
>  use crate::{
> -    gsp::cmdq::Cmdq,
> -    gsp::fw::{
> -        GspArgumentsPadded,
> -        LibosMemoryRegionInitArgument, //
> +    fwctl::NovaCoreFwCtl,
> +    gsp::{
> +        cmdq::Cmdq,
> +        fw::{
> +            GspArgumentsPadded,
> +            LibosMemoryRegionInitArgument, //
> +        },
>      },
>      num,
>  };
> @@ -117,6 +123,12 @@ pub(crate) struct Gsp {
>      pub(crate) cmdq: Cmdq,
>      /// RM arguments.
>      rmargs: CoherentAllocation<GspArgumentsPadded>,
> +    /// Cached RM internal client handle from GSP static info.
> +    pub(crate) h_client: u32,
> +    /// Cached RM internal subdevice handle from GSP static info.
> +    pub(crate) h_subdevice: u32,
> +    /// fwctl registration for userspace RM control.
> +    fwctl: Pin<KBox<Devres<fwctl::Registration<NovaCoreFwCtl>>>>,

This does not need to be in a separate allocation.

>  }
>  
>  impl Gsp {
> @@ -125,6 +137,8 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> impl PinInit<Self, Error
>          pin_init::pin_init_scope(move || {
>              let dev = pdev.as_ref();
>  
> +            let fwctl_dev = fwctl::Device::<NovaCoreFwCtl>::new(pdev.as_ref(), Ok(()))?;

You should store everything you need in the private data of your fwctl::Device
to properly tie it to its lifetime, e.g.

	struct NovaCoreFwCtl {
		gpu: Arc<Gpu>,
		bar: Arc<Devres<Bar0>>,
	}

> +
>              Ok(try_pin_init!(Self {
>                  libos: CoherentAllocation::<LibosMemoryRegionInitArgument>::alloc_coherent(
>                      dev,
> @@ -140,6 +154,9 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> impl PinInit<Self, Error
>                      1,
>                      GFP_KERNEL | __GFP_ZERO,
>                  )?,
> +                h_client: 0,
> +                h_subdevice: 0,
> +                fwctl: KBox::pin_init(fwctl::Registration::new(pdev.as_ref(), &fwctl_dev), GFP_KERNEL)?,

I don't think this should live in the Gsp structure, please move this to probe()
and store the registration in struct NovaCore after the Gpu structure.

>                  _: {
>                      // Initialise the logging structures. The OpenRM equivalents are in:
>                      // _kgspInitLibosLoggingStructures (allocates memory for buffers)
> diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
> index bc53e667cd9e..f493546b78ff 100644
> --- a/drivers/gpu/nova-core/gsp/boot.rs
> +++ b/drivers/gpu/nova-core/gsp/boot.rs
> @@ -128,7 +128,7 @@ fn run_fwsec_frts(
>      ///
>      /// Upon return, the GSP is up and running, and its runtime object given as return value.
>      pub(crate) fn boot(
> -        self: Pin<&mut Self>,
> +        mut self: Pin<&mut Self>,
>          pdev: &pci::Device<device::Bound>,
>          bar: &Bar0,
>          chipset: Chipset,
> @@ -221,6 +221,10 @@ pub(crate) fn boot(
>  
>          // Obtain and display basic GPU information.
>          let info = commands::get_gsp_info(&self.cmdq, bar)?;
> +        // SAFETY: h_client and h_subdevice are not structurally pinned.
> +        let this = unsafe { self.as_mut().get_unchecked_mut() };
> +        this.h_client = info.h_client();
> +        this.h_subdevice = info.h_subdevice();

Ick! If this is only needed from the fwctl callbacks, just move those into your
fwctl::Device private data.

But I think this is needed in other cases as well. So, more generally I think
Gsp::boot() should just be part of the constructor, as we won't be able to
properly construct a Gsp object without having a chance to interact with the Gsp
before.

>  /// Sends an RM control command, checks the reply status, and returns the raw parameter bytes.
> -fn send_rm_control(
> +pub(crate) fn send_rm_control(
>      cmdq: &Cmdq,
>      bar: &Bar0,
>      h_client: u32,
> @@ -106,7 +106,7 @@ fn send_rm_control(
>      cmd: RmControlMsgFunction,
>      params: &[u8],
>  ) -> Result<KVVec<u8>> {
> -    let reply = cmdq.send_sync_command(bar, RmControl::new(h_client, h_object, cmd, params))?;
> +    let reply = cmdq.send_command(bar, RmControl::new(h_client, h_object, cmd, params))?;

This looks like it should be a separate patch, or fixed in one of the
dependencies of this patch.

>  
>      Result::from(reply.status)?;
>  
> diff --git a/drivers/gpu/nova-core/nova_core.rs b/drivers/gpu/nova-core/nova_core.rs
> index b5caf1044697..863dc041272c 100644
> --- a/drivers/gpu/nova-core/nova_core.rs
> +++ b/drivers/gpu/nova-core/nova_core.rs
> @@ -10,6 +10,7 @@
>  mod falcon;
>  mod fb;
>  mod firmware;
> +mod fwctl;
>  mod gfw;
>  mod gpu;
>  mod gsp;
> @@ -27,6 +28,7 @@
>      description: "Nova Core GPU driver",
>      license: "GPL v2",
>      firmware: [],
> +    imports_ns: ["FWCTL"],
>  }
>  
>  kernel::module_firmware!(firmware::ModInfoBuilder);
> diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h
> index 716ac0eee42d..f6289fbf3062 100644
> --- a/include/uapi/fwctl/fwctl.h
> +++ b/include/uapi/fwctl/fwctl.h
> @@ -45,6 +45,7 @@ enum fwctl_device_type {
>  	FWCTL_DEVICE_TYPE_MLX5 = 1,
>  	FWCTL_DEVICE_TYPE_CXL = 2,
>  	FWCTL_DEVICE_TYPE_PDS = 4,
> +	FWCTL_DEVICE_TYPE_NOVA_CORE = 5,
>  };
>  
>  /**
> diff --git a/include/uapi/fwctl/nova-core.h b/include/uapi/fwctl/nova-core.h
> new file mode 100644
> index 000000000000..3f1d94b44ec8
> --- /dev/null
> +++ b/include/uapi/fwctl/nova-core.h

Why is this not include/uapi/gpu/nova-core.h? This driver does not live under
drivers/fwctl/.

If we want to separate it, I suggest include/uapi/gpu/nova-core/fwctl.h, but it
doesn't seem necessary as we don't have any other UAPIs.

Re: [PATCH 2/2] gpu: nova-core: add fwctl driver for firmware control interface

[Jason Gunthorpe]

On Fri, Mar 13, 2026 at 05:03:28PM +0100, Danilo Krummrich wrote:
> >  /**
> > diff --git a/include/uapi/fwctl/nova-core.h b/include/uapi/fwctl/nova-core.h
> > new file mode 100644
> > index 000000000000..3f1d94b44ec8
> > --- /dev/null
> > +++ b/include/uapi/fwctl/nova-core.h
> 
> Why is this not include/uapi/gpu/nova-core.h? This driver does not live under
> drivers/fwctl/.

I would prefer if it did, but understand if it doesn't want to for
reasons.

But the uapi headers still must live in uapi/fwctl/ not someplace
else, that is important.

Jason