From: Joel Fernandes <joelagnelf@nvidia.com>
To: linux-kernel@vger.kernel.org, Danilo Krummrich <dakr@kernel.org>,
David Airlie <airlied@gmail.com>, Simona Vetter <simona@ffwll.ch>,
Maarten Lankhorst <maarten.lankhorst@linux.intel.com>,
Maxime Ripard <mripard@kernel.org>,
Thomas Zimmermann <tzimmermann@suse.de>,
Jonathan Corbet <corbet@lwn.net>
Cc: nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org,
Alexandre Courbot <acourbot@nvidia.com>,
John Hubbard <jhubbard@nvidia.com>,
Shirish Baskaran <sbaskaran@nvidia.com>,
Alistair Popple <apopple@nvidia.com>,
Timur Tabi <ttabi@nvidia.com>, Ben Skeggs <bskeggs@nvidia.com>,
Joel Fernandes <joelagnelf@nvidia.com>,
linux-doc@vger.kernel.org
Subject: [PATCH 4/6] nova-core: docs: Document fwsec operation and layout
Date: Wed, 23 Apr 2025 18:54:00 -0400 [thread overview]
Message-ID: <20250423225405.139613-5-joelagnelf@nvidia.com> (raw)
In-Reply-To: <20250423225405.139613-1-joelagnelf@nvidia.com>
Add explanation of fwsec with diagrams. This helps clarify how the
nova-core falcon boot works.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---
Documentation/gpu/nova/core/fwsec.rst | 173 ++++++++++++++++++++++++++
Documentation/gpu/nova/core/vbios.rst | 3 +-
Documentation/gpu/nova/index.rst | 1 +
3 files changed, 176 insertions(+), 1 deletion(-)
create mode 100644 Documentation/gpu/nova/core/fwsec.rst
diff --git a/Documentation/gpu/nova/core/fwsec.rst b/Documentation/gpu/nova/core/fwsec.rst
new file mode 100644
index 000000000000..03d1f0d67414
--- /dev/null
+++ b/Documentation/gpu/nova/core/fwsec.rst
@@ -0,0 +1,173 @@
+.. SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+=========================
+FWSEC (Firmware Security)
+=========================
+
+The role of FWSEC to provide firmware verification and perform secure operations
+such as carving out the WPR2 region (Write protected region). It is critical to
+the boot sequence of the GPU.
+
+The FWSEC image is located in the VBIOS ROM. For how it is extracted, see vbios.rst
+and the vbios.rs source code.
+
+The Falcon data in the FWSEC image is a combination of headers, data sections
+(DMEM) and instruction code sections (IMEM).
+
+FWSEC itself is a central location for all microcodes in the VBIOS that run on
+various microcontrollers (PMU, SEC, GSP etc) some of which (such as devinit) are
+executed after GPU power-on before the nova-core driver is even loaded.
+
+For the purposes of nova-core driver, the FWSEC contains microcode called
+DMEMMAPPER which executes the "FWSEC-FRTS" command (among other commands it is
+capable of executing) that is executed on the GSP to carve out the WPR2 region
+(Write protected region).
+
+FWSEC Memory Layout
+-------------------
+
+The memory layout of the FWSEC image is as follows (this is using an GA-102
+Ampere GPU as an example and could vary for future GPUs):
+
+┌───────────────────────────────────────────────────────────────┐
+│ FWSEC image (type 0xE0) │
+│ │
+│ ┌─────────────────────────────────┐ │
+│ │ PMU Falcon Ucode Table │ │
+│ │ (PmuLookupTable) │ │
+│ │ ┌─────────────────────────┐ │ │
+│ │ │ Table Header │ │ │
+│ │ │ - version: 0x01 │ │ │
+│ │ │ - header_size: 6 │ │ │
+│ │ │ - entry_size: 6 │ │ │
+│ │ │ - entry_count: N │ │ │
+│ │ │ - desc_version:3(unused)│ │ │
+│ │ └─────────────────────────┘ │ │
+│ │ ... │ │
+│ │ ┌─────────────────────────┐ │ │
+│ │ │ Entry for FWSEC (0x85) │ │ │
+│ │ │ (PmuLookupTableEntry) │ │ │
+│ │ │ - app_id: 0x85 (FWSEC) │ ───┼────┐ │
+│ │ │ - target_id: 0x01 (PMU) │ │ │ │
+│ │ │ - data: offset ─────────┼────┼────┼───┐ │
+│ │ └─────────────────────────┘ │ │ │ │
+│ └─────────────────────────────────┘ │ │ │
+│ │ │ │
+│ │ │ │
+│ ┌─────────────────────────────────┐ │ │ │
+│ │ FWSEC Ucode Component │<───┘ │ │
+│ │ (aka Falcon data) │ │ │
+│ │ ┌─────────────────────────┐ │ │ │
+│ │ │ FalconUCodeDescV3 │<───┼────────┘ │
+│ │ │ - hdr │ │ │
+│ │ │ - stored_size │ │ │
+│ │ │ - pkc_data_offset │ │ │
+│ │ │ - interface_offset ─────┼────┼────────────────┐ │
+│ │ │ - imem_phys_base │ │ │ │
+│ │ │ - imem_load_size │ │ │ │
+│ │ │ - imem_virt_base │ │ │ │
+│ │ │ - dmem_phys_base │ │ │ │
+│ │ │ - dmem_load_size │ │ │ │
+│ │ │ - engine_id_mask │ │ │ │
+│ │ │ - ucode_id │ │ │ │
+│ │ │ - signature_count │ │ look up sig │ │
+│ │ │ - signature_versions ───┼────┼─────┐using hal │ │
+│ │ └─────────────────────────┘ │ │ │ │
+│ │ (no gap) │ │ │ │
+│ │ ┌─────────────────────────┐ │ │ │ │
+│ │ │ Signatures Section │<───┼─────┘ │ │
+│ │ │ (384 bytes per sig) │ │ │ │
+│ │ │ - RSA-3K Signature 1 │ │ │ │
+│ │ │ - RSA-3K Signature 2 │ │ │ │
+│ │ │ ... │ │ │ │
+│ │ └─────────────────────────┘ │ │ │
+│ │ │ │ │
+│ │ ┌─────────────────────────┐ │ │ │
+│ │ │ IMEM Section (Code) │ │ │ │
+│ │ │ │ │ │ │
+│ │ │ Contains devinit, fwsec │ │ │ │
+│ │ │ instruction code etc. │ │ │ │
+│ │ └─────────────────────────┘ │ │ │
+│ │ │ │ │
+│ │ ┌─────────────────────────┐ │ │ │
+│ │ │ DMEM Section (Data) │ │ │ │
+│ │ │ │ │ │ │
+│ │ │ ┌─────────────────────┐ │ │ │ │
+│ │ │ │ Application │ │<───┼────────────────┘ │
+│ │ │ │ Interface Table │ │ │ │
+│ │ │ │ (FalconAppifHdrV1) │ │ │ │
+│ │ │ │ Header: │ │ │ │
+│ │ │ │ - version: 0x01 │ │ │ │
+│ │ │ │ - header_size: 4 │ │ │ │
+│ │ │ │ - entry_size: 8 │ │ │ │
+│ │ │ │ - entry_count: N │ │ │ │
+│ │ │ │ │ │ │ │
+│ │ │ │ Entries: │ │ │ │
+│ │ │ │ ┌─────────────────┐ │ │ │ │
+│ │ │ │ │ DEVINIT (ID 1) │ │ │ │ │
+│ │ │ │ │ - id: 0x01 │ │ │ │ │
+│ │ │ │ │ - dmemOffset X ─┼─┼─┼────┐ │
+│ │ │ │ └─────────────────┘ │ │ │ │
+│ │ │ │ ┌─────────────────┐ │ │ │ │
+│ │ │ │ │ DMEMMAPPER(ID 4)│ │ │ │ │
+│ │ │ │ │ - id: 0x04 │ │ │ │ │
+| | | | | (NVFW_FALCON_ | | | | |
+| | | | | APPIF_ID_DMEMMAPPER) | |
+│ │ │ │ │ - dmemOffset Y ─┼─┼─┼────┼─────┐ │
+│ │ │ │ └─────────────────┘ │ │ │ │ │
+│ │ │ │ ┌─────────────────┐ │ │ │ │ │
+│ │ │ │ │ MULTI_FALCON (5)│ │ │ │ │ │
+│ │ │ │ │ - id: 0x05 │ │ │ │ │ (See note [1]) │
+│ │ │ │ │ - dmemOffset Z ─┼─┼─┼────┼─────┼────────────────┐ │
+│ │ │ │ └─────────────────┘ │ │ │ │ │ │
+│ │ │ └─────────────────────┘ │ │ │ │ │
+│ │ │ │ │ │ │ │
+│ │ │ ┌─────────────────────┐ │ │ │ │ │
+│ │ │ │ DEVINIT Engine │<┼────┘ │ │ │
+│ │ │ │ Interface │ │ │ │ │ │
+│ │ │ └─────────────────────┘ │ │ │ │ │
+│ │ │ │ │ │ │ │
+│ │ │ ┌─────────────────────┐ │ │ │ │ │
+│ │ │ │ DMEM Mapper (ID 4) │<┼────+─────┘ │ │
+│ │ │ │ (FalconAppifDmemmapperV3) | | |
+│ │ │ │ - signature: "DMAP" │ │ │ | |
+│ │ │ │ - version: 0x0003 │ │ │ | |
+│ │ │ │ - Size: 64 bytes │ │ │ | |
+│ │ │ │ - cmd_in_buffer_off │ │────┼────────────┐ | |
+│ │ │ │ - cmd_in_buffer_size│ │ │ │ | |
+│ │ │ │ - cmd_out_buffer_off│ │────┼────────────┼─────┐ | |
+│ │ │ │ - cmd_out_buffer_sz │ │ │ │ │ | |
+│ │ │ │ - init_cmd │ │ │ │ │ | |
+│ │ │ │ - features │ │ │ │ │ | |
+│ │ │ │ - cmd_mask0/1 │ │ │ │ │ V |
+│ │ │ │ - multiTgtTbl-------┼─┼────+────────────+─────+────┐ │
+│ │ │ └─────────────────────┘ │ │ │ │ │ │
+│ │ │ │ │ │ │ │ │
+│ │ │ ┌─────────────────────┐ │ │ │ │ │ │
+│ │ │ │ MULTI_FALCON │<┼────+────────────+─────+────┘ │
+│ │ │ │ - version │ │ │ │ │ |
+│ │ │ │ - targetId (PMU/SEC)│ │ │ │ │ |
+│ │ │ │ - loadType │ │ │ │ │ |
+│ │ │ │ - initStack │ │ │ │ │ |
+│ │ │ └─────────────────────┘ │ │ │ │ |
+│ │ │ │ │ │ │ |
+│ │ │ ┌─────────────────────┐ │ │ │ │ |
+│ │ │ │ Command Input Buffer│<┼────┼────────────┘ │ |
+│ │ │ │ - Command data │ │ │ │ |
+│ │ │ │ - Arguments │ │ │ │ |
+│ │ │ └─────────────────────┘ │ │ │ |
+│ │ │ │ │ │ |
+│ │ │ ┌─────────────────────┐ │ │ │ |
+│ │ │ │ Command Output │<┼────┼──────────────────┘ |
+│ │ │ │ Buffer │ │ │ |
+│ │ │ │ - Results │ │ │ |
+│ │ │ │ - Status │ │ │ |
+│ │ │ └─────────────────────┘ │ │ |
+│ │ └─────────────────────────┘ │ |
+│ └─────────────────────────────────┘ |
+│ │
+└───────────────────────────────────────────────────────────────┘
+
+Note:
+[1] MULTI_FALCON section in DMEM can either be looked up by
+ MULTI_FALCON App Interface, or from multi_tgt_table entry
+ in DMEM_MAPPER's App Interface.
\ No newline at end of file
diff --git a/Documentation/gpu/nova/core/vbios.rst b/Documentation/gpu/nova/core/vbios.rst
index 17411f21b410..da759b56fb2d 100644
--- a/Documentation/gpu/nova/core/vbios.rst
+++ b/Documentation/gpu/nova/core/vbios.rst
@@ -44,7 +44,8 @@ VBIOS ROM Layout
The VBIOS layout is roughly a series of concatenated images as follows:
(For more explanations of acronyms, see the detailed descriptions in vbios.rs).
-
+Note: this diagram is using an GA-102 Ampere GPU as an example and could vary
+ for future GPUs.
┌────────────────────────────────────────────────────────────────────────┐
│ VBIOS (Starting at ROM_OFFSET: 0x300000) │
├────────────────────────────────────────────────────────────────────────┤
diff --git a/Documentation/gpu/nova/index.rst b/Documentation/gpu/nova/index.rst
index 2701b3f4af35..c01dcc5657e2 100644
--- a/Documentation/gpu/nova/index.rst
+++ b/Documentation/gpu/nova/index.rst
@@ -27,4 +27,5 @@ vGPU manager VFIO driver and the nova-drm driver.
:titlesonly:
core/guidelines
+ core/fwsec
core/todo
--
2.43.0
next prev parent reply other threads:[~2025-04-23 22:54 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-04-23 22:53 [PATCH 0/6] Additional documentation for nova-core Joel Fernandes
2025-04-23 22:53 ` [PATCH 1/6] nova-core: doc: Add code comments related to devinit Joel Fernandes
2025-04-23 22:53 ` [PATCH 2/6] nova-core: doc: Clarify sysmembar operations Joel Fernandes
2025-04-23 22:53 ` [PATCH 3/6] nova-core: docs: Document vbios layout Joel Fernandes
2025-04-23 23:12 ` Timur Tabi
2025-04-23 23:30 ` Joel Fernandes
2025-04-23 22:54 ` Joel Fernandes [this message]
2025-04-23 22:54 ` [PATCH 5/6] gpu: nova-core: Clarify fields in FalconAppifHdrV1 Joel Fernandes
2025-04-24 1:18 ` Alexandre Courbot
[not found] ` <174546399398.876.3516508778193165894@patchwork.local>
2025-04-24 7:45 ` [5/6] " Alexandre Courbot
2025-04-24 13:03 ` Joel Fernandes
2025-04-23 22:54 ` [PATCH 6/6] nova-core: docs: Document devinit process Joel Fernandes
2025-04-24 1:25 ` [PATCH 0/6] Additional documentation for nova-core Alexandre Courbot
2025-04-24 8:10 ` Danilo Krummrich
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250423225405.139613-5-joelagnelf@nvidia.com \
--to=joelagnelf@nvidia.com \
--cc=acourbot@nvidia.com \
--cc=airlied@gmail.com \
--cc=apopple@nvidia.com \
--cc=bskeggs@nvidia.com \
--cc=corbet@lwn.net \
--cc=dakr@kernel.org \
--cc=dri-devel@lists.freedesktop.org \
--cc=jhubbard@nvidia.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=maarten.lankhorst@linux.intel.com \
--cc=mripard@kernel.org \
--cc=nouveau@lists.freedesktop.org \
--cc=sbaskaran@nvidia.com \
--cc=simona@ffwll.ch \
--cc=ttabi@nvidia.com \
--cc=tzimmermann@suse.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox