* [PATCH 1/5] gpu: nova-core: add Tlv.get_u64
2026-11-10 23:10 [PATCH 0/5] Transition nova-core to use TLV variables for heap sizes Timur Tabi
@ 2026-11-10 23:10 ` Timur Tabi
2026-11-11 0:20 ` John Hubbard
2026-11-11 5:27 ` Eliot Courtney
2026-11-10 23:10 ` [PATCH 2/5] gpu: nova-core: merge LibosParams into GspFwHeapParams Timur Tabi
` (4 subsequent siblings)
5 siblings, 2 replies; 13+ messages in thread
From: Timur Tabi @ 2026-11-10 23:10 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot, John Hubbard, Eliot Courtney,
nova-gpu
Introduce get_u64() to support variable-sized TLV integers. If the driver
expects that a particular tag might be a 64-bit value, it should use this
method read it.
TLVs can store unsigned integers in either 32-bit or 64-bit little-endian
format. The script that generates TLV files is written in Python, and
Python integers are arbitrarily sized. For any given integer, the script
will store it as a 32-bit integer if its value is less than 0x100000000,
without any regard to its purpose. This means that the driver may not
know until runtime whether a given TLV integer is encoded as four bytes
or eight.
Signed-off-by: Timur Tabi <ttabi@nvidia.com>
---
drivers/gpu/nova-core/firmware/tlv.rs | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/drivers/gpu/nova-core/firmware/tlv.rs b/drivers/gpu/nova-core/firmware/tlv.rs
index 7f278903dc8b..75d8f907421b 100644
--- a/drivers/gpu/nova-core/firmware/tlv.rs
+++ b/drivers/gpu/nova-core/firmware/tlv.rs
@@ -257,6 +257,23 @@ pub(crate) fn load_file(
.ok_or(EINVAL)
}
+ /// Return a u64, from either a 32-bit or 64-bit little-endian tag.
+ #[expect(unused)]
+ pub(crate) fn get_u64(&self, tag: &[u8; 4]) -> Result<u64> {
+ let value = self.get_bytes(tag)?;
+
+ match value.len() {
+ // `value` is an unsized slice, but from_le_bytes() requires a sized
+ // slice (&[u8; N]), so we need to convert.
+ 4 => value
+ .first_chunk()
+ .map(|&b| u64::from(u32::from_le_bytes(b))),
+ 8 => value.first_chunk().map(|&b| u64::from_le_bytes(b)),
+ _ => None,
+ }
+ .ok_or(EINVAL)
+ }
+
/// Return a string value.
pub(crate) fn get_string(&self, tag: &[u8; 4]) -> Result<&'a str> {
let tlv = self.find(tag)?;
--
2.54.0
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH 1/5] gpu: nova-core: add Tlv.get_u64
2026-11-10 23:10 ` [PATCH 1/5] gpu: nova-core: add Tlv.get_u64 Timur Tabi
@ 2026-11-11 0:20 ` John Hubbard
2026-11-11 2:59 ` Timur Tabi
2026-11-11 5:27 ` Eliot Courtney
1 sibling, 1 reply; 13+ messages in thread
From: John Hubbard @ 2026-11-11 0:20 UTC (permalink / raw)
To: Timur Tabi, Danilo Krummrich, Alexandre Courbot, Eliot Courtney,
nova-gpu
On 11/10/26 3:10 PM, Timur Tabi wrote:
...
> diff --git a/drivers/gpu/nova-core/firmware/tlv.rs b/drivers/gpu/nova-core/firmware/tlv.rs
> index 7f278903dc8b..75d8f907421b 100644
> --- a/drivers/gpu/nova-core/firmware/tlv.rs
> +++ b/drivers/gpu/nova-core/firmware/tlv.rs
> @@ -257,6 +257,23 @@ pub(crate) fn load_file(
> .ok_or(EINVAL)
> }
>
> + /// Return a u64, from either a 32-bit or 64-bit little-endian tag.
> + #[expect(unused)]
> + pub(crate) fn get_u64(&self, tag: &[u8; 4]) -> Result<u64> {
> + let value = self.get_bytes(tag)?;
> +
> + match value.len() {
> + // `value` is an unsized slice, but from_le_bytes() requires a sized
> + // slice (&[u8; N]), so we need to convert.
> + 4 => value
At this point, we have established the length...
> + .first_chunk()
...but then first_chunk() checks the length again, and also returns
an Option that is never None.
> + .map(|&b| u64::from(u32::from_le_bytes(b))),
> + 8 => value.first_chunk().map(|&b| u64::from_le_bytes(b)),
> + _ => None,
> + }
> + .ok_or(EINVAL)
> + }
Instead, we could use try_from() to check the length and convert
the slice, in one step. Your TlvBlockHeader::parse() already uses
that approach, and I think it would also work here.
Something like this:
pub(crate) fn get_u64(&self, tag: &[u8; 4]) -> Result<u64> {
let value = self.get_bytes(tag)?;
if let Ok(bytes) = <[u8; 4]>::try_from(value) {
Ok(u64::from(u32::from_le_bytes(bytes)))
} else if let Ok(bytes) = <[u8; 8]>::try_from(value) {
Ok(u64::from_le_bytes(bytes))
} else {
Err(EINVAL)
}
}
thanks,
--
John Hubbard
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH 1/5] gpu: nova-core: add Tlv.get_u64
2026-11-11 0:20 ` John Hubbard
@ 2026-11-11 2:59 ` Timur Tabi
2026-11-11 3:45 ` John Hubbard
0 siblings, 1 reply; 13+ messages in thread
From: Timur Tabi @ 2026-11-11 2:59 UTC (permalink / raw)
To: nova-gpu@lists.linux.dev, Alexandre Courbot, dakr@kernel.org,
Eliot Courtney, John Hubbard
On Tue, 2026-11-10 at 16:20 -0800, John Hubbard wrote:
> On 11/10/26 3:10 PM, Timur Tabi wrote:
> ...
> > diff --git a/drivers/gpu/nova-core/firmware/tlv.rs b/drivers/gpu/nova-core/firmware/tlv.rs
> > index 7f278903dc8b..75d8f907421b 100644
> > --- a/drivers/gpu/nova-core/firmware/tlv.rs
> > +++ b/drivers/gpu/nova-core/firmware/tlv.rs
> > @@ -257,6 +257,23 @@ pub(crate) fn load_file(
> > .ok_or(EINVAL)
> > }
> >
> > + /// Return a u64, from either a 32-bit or 64-bit little-endian tag.
> > + #[expect(unused)]
> > + pub(crate) fn get_u64(&self, tag: &[u8; 4]) -> Result<u64> {
> > + let value = self.get_bytes(tag)?;
> > +
> > + match value.len() {
> > + // `value` is an unsized slice, but from_le_bytes() requires a sized
> > + // slice (&[u8; N]), so we need to convert.
> > + 4 => value
>
>
> At this point, we have established the length...
>
>
> > + .first_chunk()
>
>
> ...but then first_chunk() checks the length again, and also returns
> an Option that is never None.
In an optimized build, that check is not actually done. The only checks are for the match
statement.
cmpq $8, %rsi
je .LBB0_4
cmpq $4, %rsi
jne .LBB0_2
movl (%rdi), %edx
movl $1, %eax
retq
.LBB0_4:
movq (%rdi), %rdx
movl $1, %eax
retq
.LBB0_2:
xorl %eax, %eax
retq
Of course, it's still terrible compared to the C version.
> > + .map(|&b| u64::from(u32::from_le_bytes(b))),
> > + 8 => value.first_chunk().map(|&b| u64::from_le_bytes(b)),
> > + _ => None,
> > + }
> > + .ok_or(EINVAL)
> > + }
>
> Instead, we could use try_from() to check the length and convert
> the slice, in one step. Your TlvBlockHeader::parse() already uses
> that approach, and I think it would also work here.
>
> Something like this:
>
> pub(crate) fn get_u64(&self, tag: &[u8; 4]) -> Result<u64> {
> let value = self.get_bytes(tag)?;
>
> if let Ok(bytes) = <[u8; 4]>::try_from(value) {
> Ok(u64::from(u32::from_le_bytes(bytes)))
> } else if let Ok(bytes) = <[u8; 8]>::try_from(value) {
> Ok(u64::from_le_bytes(bytes))
> } else {
> Err(EINVAL)
> }
> }
I'm more than happy to change it to whatever other people want. I went through several
iterations of this code, working with Cursor to evaluate each one. IMHO, they are all ugly and
hard to read, because of the from_le_bytes() constraint that I mentioned in the comment.
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH 1/5] gpu: nova-core: add Tlv.get_u64
2026-11-11 2:59 ` Timur Tabi
@ 2026-11-11 3:45 ` John Hubbard
2026-11-11 5:21 ` Eliot Courtney
0 siblings, 1 reply; 13+ messages in thread
From: John Hubbard @ 2026-11-11 3:45 UTC (permalink / raw)
To: Timur Tabi, nova-gpu@lists.linux.dev, Alexandre Courbot,
dakr@kernel.org, Eliot Courtney
On 11/10/26 6:59 PM, Timur Tabi wrote:
> On Tue, 2026-11-10 at 16:20 -0800, John Hubbard wrote:
>> On 11/10/26 3:10 PM, Timur Tabi wrote:
>> ...
>> ...but then first_chunk() checks the length again, and also returns
>> an Option that is never None.
>
> In an optimized build, that check is not actually done. The only checks are for the match
> statement.
Nice. OK.
...
>> Instead, we could use try_from() to check the length and convert
>> the slice, in one step. Your TlvBlockHeader::parse() already uses
>> that approach, and I think it would also work here.
>>
>> Something like this:
>>
>> pub(crate) fn get_u64(&self, tag: &[u8; 4]) -> Result<u64> {
>> let value = self.get_bytes(tag)?;
>>
>> if let Ok(bytes) = <[u8; 4]>::try_from(value) {
>> Ok(u64::from(u32::from_le_bytes(bytes)))
>> } else if let Ok(bytes) = <[u8; 8]>::try_from(value) {
>> Ok(u64::from_le_bytes(bytes))
>> } else {
>> Err(EINVAL)
>> }
>> }
>
> I'm more than happy to change it to whatever other people want. I went through several
> iterations of this code, working with Cursor to evaluate each one. IMHO, they are all ugly and
> hard to read, because of the from_le_bytes() constraint that I mentioned in the comment.
The reason I responded was that I think the try_from() is nicer
on-screen. I hadn't gone all the way to the asm code.
thanks,
--
John Hubbard
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH 1/5] gpu: nova-core: add Tlv.get_u64
2026-11-11 3:45 ` John Hubbard
@ 2026-11-11 5:21 ` Eliot Courtney
0 siblings, 0 replies; 13+ messages in thread
From: Eliot Courtney @ 2026-11-11 5:21 UTC (permalink / raw)
To: John Hubbard, Timur Tabi, nova-gpu@lists.linux.dev,
Alexandre Courbot, dakr@kernel.org, Eliot Courtney
On Wed Nov 11, 2026 at 12:45 PM JST, John Hubbard wrote:
> On 11/10/26 6:59 PM, Timur Tabi wrote:
>> On Tue, 2026-11-10 at 16:20 -0800, John Hubbard wrote:
>>> On 11/10/26 3:10 PM, Timur Tabi wrote:
>>> ...
>>> ...but then first_chunk() checks the length again, and also returns
>>> an Option that is never None.
>>
>> In an optimized build, that check is not actually done. The only checks are for the match
>> statement.
>
> Nice. OK.
>
> ...
>>> Instead, we could use try_from() to check the length and convert
>>> the slice, in one step. Your TlvBlockHeader::parse() already uses
>>> that approach, and I think it would also work here.
>>>
>>> Something like this:
>>>
>>> pub(crate) fn get_u64(&self, tag: &[u8; 4]) -> Result<u64> {
>>> let value = self.get_bytes(tag)?;
>>>
>>> if let Ok(bytes) = <[u8; 4]>::try_from(value) {
>>> Ok(u64::from(u32::from_le_bytes(bytes)))
>>> } else if let Ok(bytes) = <[u8; 8]>::try_from(value) {
>>> Ok(u64::from_le_bytes(bytes))
>>> } else {
>>> Err(EINVAL)
>>> }
>>> }
>>
>> I'm more than happy to change it to whatever other people want. I went through several
>> iterations of this code, working with Cursor to evaluate each one. IMHO, they are all ugly and
>> hard to read, because of the from_le_bytes() constraint that I mentioned in the comment.
>
> The reason I responded was that I think the try_from() is nicer
> on-screen. I hadn't gone all the way to the asm code.
>
> thanks,
I prefer John's version which looks a bit more readable to me. It's also
possible to write it like this to avoid writing the array types,
although I don't think it's necessarily better:
```
let value = self.get_bytes(tag)?;
if let Ok(value) = value.try_into().map(u32::from_le_bytes) {
Ok(u64::from(value))
} else if let Ok(value) = value.try_into().map(u64::from_le_bytes) {
Ok(value)
} else {
Err(EINVAL)
}
```
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 1/5] gpu: nova-core: add Tlv.get_u64
2026-11-10 23:10 ` [PATCH 1/5] gpu: nova-core: add Tlv.get_u64 Timur Tabi
2026-11-11 0:20 ` John Hubbard
@ 2026-11-11 5:27 ` Eliot Courtney
1 sibling, 0 replies; 13+ messages in thread
From: Eliot Courtney @ 2026-11-11 5:27 UTC (permalink / raw)
To: Timur Tabi, Danilo Krummrich, Alexandre Courbot, John Hubbard,
Eliot Courtney, nova-gpu
On Wed Nov 11, 2026 at 8:10 AM JST, Timur Tabi wrote:
> Introduce get_u64() to support variable-sized TLV integers. If the driver
> expects that a particular tag might be a 64-bit value, it should use this
> method read it.
>
> TLVs can store unsigned integers in either 32-bit or 64-bit little-endian
> format. The script that generates TLV files is written in Python, and
> Python integers are arbitrarily sized. For any given integer, the script
> will store it as a 32-bit integer if its value is less than 0x100000000,
> without any regard to its purpose. This means that the driver may not
> know until runtime whether a given TLV integer is encoded as four bytes
> or eight.
>
> Signed-off-by: Timur Tabi <ttabi@nvidia.com>
> ---
> drivers/gpu/nova-core/firmware/tlv.rs | 17 +++++++++++++++++
> 1 file changed, 17 insertions(+)
>
> diff --git a/drivers/gpu/nova-core/firmware/tlv.rs b/drivers/gpu/nova-core/firmware/tlv.rs
> index 7f278903dc8b..75d8f907421b 100644
> --- a/drivers/gpu/nova-core/firmware/tlv.rs
> +++ b/drivers/gpu/nova-core/firmware/tlv.rs
> @@ -257,6 +257,23 @@ pub(crate) fn load_file(
> .ok_or(EINVAL)
> }
>
> + /// Return a u64, from either a 32-bit or 64-bit little-endian tag.
> + #[expect(unused)]
> + pub(crate) fn get_u64(&self, tag: &[u8; 4]) -> Result<u64> {
> + let value = self.get_bytes(tag)?;
`get_u32` is using `self.find`, but this is using `self.get_bytes`.
Probably they should use the same one, particularly since when the TODO
to use ENODATA is done the error behaviour would differ between these
two functions.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 2/5] gpu: nova-core: merge LibosParams into GspFwHeapParams
2026-11-10 23:10 [PATCH 0/5] Transition nova-core to use TLV variables for heap sizes Timur Tabi
2026-11-10 23:10 ` [PATCH 1/5] gpu: nova-core: add Tlv.get_u64 Timur Tabi
@ 2026-11-10 23:10 ` Timur Tabi
2026-11-11 6:30 ` Eliot Courtney
2026-11-10 23:10 ` [PATCH 3/5] gpu: nova-core: read GSP heap variables from TLV Timur Tabi
` (3 subsequent siblings)
5 siblings, 1 reply; 13+ messages in thread
From: Timur Tabi @ 2026-11-10 23:10 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot, John Hubbard, Eliot Courtney,
nova-gpu
LibosParams and GspFwHeapParams are two types used to hold data and/or
methods for calculating the heap sides that GSP-RM needs for booting.
Although they technically represent distinct parts of the equations,
they are always used together.
A future change to this code will introduce runtime calculations based
on values extracted from the GSP-RM, replacing the compile-time
constants currently used. Merging the two types in this commit keeps
the current functionality while providing a foundation that simplifies
future changes.
For the same reason, the LIBOSx constants are replaced with variables.
Signed-off-by: Timur Tabi <ttabi@nvidia.com>
---
drivers/gpu/nova-core/fb.rs | 9 ++-
drivers/gpu/nova-core/gsp.rs | 4 +-
drivers/gpu/nova-core/gsp/fw.rs | 119 +++++++++++++++++---------------
3 files changed, 69 insertions(+), 63 deletions(-)
diff --git a/drivers/gpu/nova-core/fb.rs b/drivers/gpu/nova-core/fb.rs
index b0c0b8fe6008..9a3314768385 100644
--- a/drivers/gpu/nova-core/fb.rs
+++ b/drivers/gpu/nova-core/fb.rs
@@ -299,14 +299,13 @@ pub(crate) fn wpr2_range(bar: Bar0<'_>) -> Option<Range<u64>> {
/// Computes the number of VF partitions and the WPR2 heap size from the vGPU state.
fn wpr2_heap_params(chipset: Chipset, vgpu_state: VgpuState, fb_size: u64) -> Result<(u8, u64)> {
+ let heap_params = gsp::GspFwHeapParams::from_chipset(chipset);
+
Ok(match vgpu_state {
- VgpuState::Disabled => (
- 0,
- gsp::LibosParams::from_chipset(chipset).wpr_heap_size(chipset, fb_size)?,
- ),
+ VgpuState::Disabled => (0, heap_params.wpr_heap_size(fb_size)?),
VgpuState::Enabled { total_vfs } => (
u8::try_from(total_vfs.get()).map_err(|_| EINVAL)?,
- gsp::LibosParams::vgpu_wpr_heap_size(),
+ heap_params.vgpu_wpr_heap_size(),
),
})
}
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index 908d4ea72ad1..85ef94a8b5c7 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -29,9 +29,9 @@
pub(crate) use fw::{
GspFmcBootParams,
+ GspFwHeapParams,
GspFwWprMeta,
- LibosMemoryRegionInitArgument,
- LibosParams, //
+ LibosMemoryRegionInitArgument, //
};
pub(crate) use hal::boot_firmware_files;
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index c396daf06f1e..f30819fd89c2 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -65,26 +65,73 @@
/// build-time constant, so the driver chooses it, and this constant is the one copy.
pub(crate) const GSP_MSG_QUEUE_ELEMENT_SIZE_MAX: usize = GSP_PAGE_SIZE * 16;
-/// Empty type to group methods related to heap parameters for running the GSP firmware.
-enum GspFwHeapParams {}
+/// Heap memory requirements and constraints for booting GSP-RM.
+pub(crate) struct GspFwHeapParams {
+ /// The amount of heap memory used during GSP-RM boot and initialization (up to and including
+ /// the first client subdevice allocation), in bytes.
+ base_rm_size: u64,
+ /// The base amount of heap required by the GSP operating system, in bytes.
+ carveout_size: u64,
+ /// The minimum and maximum sizes allowed for the GSP FW heap, in bytes.
+ allowed_heap_size: Range<u64>,
+}
/// Minimum required alignment for the GSP heap.
const GSP_HEAP_ALIGNMENT: Alignment = Alignment::new::<{ 1 << 20 }>();
impl GspFwHeapParams {
- /// Returns the amount of GSP-RM heap memory used during GSP-RM boot and initialization (up to
- /// and including the first client subdevice allocation).
- fn base_rm_size(chipset: Chipset) -> u64 {
+ /// Version 2 of the GSP LIBOS (Turing and GA100)
+ fn libos2() -> Self {
+ Self {
+ base_rm_size: cv!(bindings::GSP_FW_HEAP_PARAM_BASE_RM_SIZE_TU10X),
+ carveout_size: cv!(bindings::GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS2),
+ allowed_heap_size: cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MIN_MB => u64)
+ * u64::SZ_1M
+ ..cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB => u64) * u64::SZ_1M,
+ }
+ }
+
+ /// Version 3 of the GSP LIBOS (GA102 - AD10x)
+ fn libos3() -> Self {
+ Self {
+ base_rm_size: cv!(bindings::GSP_FW_HEAP_PARAM_BASE_RM_SIZE_TU10X),
+ carveout_size: cv!(bindings::GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS3_BAREMETAL),
+ allowed_heap_size: cv!(
+ bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MIN_MB => u64)
+ * u64::SZ_1M
+ ..cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MAX_MB => u64)
+ * u64::SZ_1M,
+ }
+ }
+
+ /// Version 3 of the GSP LIBOS, (GH100+)
+ fn libos3_gh100() -> Self {
+ let base = Self::libos3();
+
+ Self {
+ base_rm_size: cv!(bindings::GSP_FW_HEAP_PARAM_BASE_RM_SIZE_GH100),
+ carveout_size: base.carveout_size,
+ allowed_heap_size: base.allowed_heap_size,
+ }
+ }
+
+ /// Returns the heap parameters corresponding to `chipset`.
+ ///
+ /// The parameters reported by GSP-RM are initialized to zero.
+ pub(crate) fn from_chipset(chipset: Chipset) -> Self {
match chipset.arch() {
- Architecture::Turing | Architecture::Ampere | Architecture::Ada => {
- u64::from(bindings::GSP_FW_HEAP_PARAM_BASE_RM_SIZE_TU10X)
- }
- Architecture::Hopper | Architecture::BlackwellGB10x | Architecture::BlackwellGB20x => {
- u64::from(bindings::GSP_FW_HEAP_PARAM_BASE_RM_SIZE_GH100)
- }
+ Architecture::Turing => Self::libos2(),
+ Architecture::Ampere if chipset < Chipset::GA102 => Self::libos2(),
+ Architecture::Ampere | Architecture::Ada => Self::libos3(),
+ _ => Self::libos3_gh100(),
}
}
+ /// Returns the WPR heap size to reserve when vGPU is enabled.
+ pub(crate) fn vgpu_wpr_heap_size(&self) -> u64 {
+ u64::from(bindings::GSP_FW_HEAP_SIZE_VGPU_DEFAULT)
+ }
+
/// Returns the amount of heap memory required to support a single channel allocation.
fn client_alloc_size() -> u64 {
u64::from(bindings::GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE)
@@ -103,60 +150,20 @@ fn management_overhead(fb_size: u64) -> Result<u64> {
.align_up(GSP_HEAP_ALIGNMENT)
.ok_or(EINVAL)
}
-}
-
-/// Heap memory requirements and constraints for a given version of the GSP LIBOS.
-pub(crate) struct LibosParams {
- /// The base amount of heap required by the GSP operating system, in bytes.
- carveout_size: u64,
- /// The minimum and maximum sizes allowed for the GSP FW heap, in bytes.
- allowed_heap_size: Range<u64>,
-}
-
-impl LibosParams {
- /// Version 2 of the GSP LIBOS (Turing and GA100)
- const LIBOS2: LibosParams = LibosParams {
- carveout_size: cv!(bindings::GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS2),
- allowed_heap_size: cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MIN_MB => u64)
- * u64::SZ_1M
- ..cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB => u64) * u64::SZ_1M,
- };
-
- /// Version 3 of the GSP LIBOS (GA102+)
- const LIBOS3: LibosParams = LibosParams {
- carveout_size: cv!(bindings::GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS3_BAREMETAL),
- allowed_heap_size: cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MIN_MB => u64)
- * u64::SZ_1M
- ..cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MAX_MB => u64) * u64::SZ_1M,
- };
-
- /// Returns the libos parameters corresponding to `chipset`.
- pub(crate) fn from_chipset(chipset: Chipset) -> &'static LibosParams {
- if chipset < Chipset::GA102 {
- &Self::LIBOS2
- } else {
- &Self::LIBOS3
- }
- }
-
- /// Returns the WPR heap size to reserve when vGPU is enabled.
- pub(crate) fn vgpu_wpr_heap_size() -> u64 {
- u64::from(bindings::GSP_FW_HEAP_SIZE_VGPU_DEFAULT)
- }
/// Returns the amount of memory (in bytes) to allocate for the WPR heap for a framebuffer size
- /// of `fb_size` (in bytes) for `chipset`.
- pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> Result<u64> {
+ /// of `fb_size` (in bytes).
+ pub(crate) fn wpr_heap_size(&self, fb_size: u64) -> Result<u64> {
// The WPR heap will contain the following:
// LIBOS carveout,
Ok(self
.carveout_size
// RM boot working memory,
- .saturating_add(GspFwHeapParams::base_rm_size(chipset))
+ .saturating_add(self.base_rm_size)
// One RM client,
- .saturating_add(GspFwHeapParams::client_alloc_size())
+ .saturating_add(Self::client_alloc_size())
// Overhead for memory management.
- .saturating_add(GspFwHeapParams::management_overhead(fb_size)?)
+ .saturating_add(Self::management_overhead(fb_size)?)
// Clamp to the supported heap sizes.
.clamp(self.allowed_heap_size.start, self.allowed_heap_size.end - 1))
}
--
2.54.0
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH 2/5] gpu: nova-core: merge LibosParams into GspFwHeapParams
2026-11-10 23:10 ` [PATCH 2/5] gpu: nova-core: merge LibosParams into GspFwHeapParams Timur Tabi
@ 2026-11-11 6:30 ` Eliot Courtney
0 siblings, 0 replies; 13+ messages in thread
From: Eliot Courtney @ 2026-11-11 6:30 UTC (permalink / raw)
To: Timur Tabi, Danilo Krummrich, Alexandre Courbot, John Hubbard,
Eliot Courtney, nova-gpu
On Wed Nov 11, 2026 at 8:10 AM JST, Timur Tabi wrote:
> LibosParams and GspFwHeapParams are two types used to hold data and/or
> methods for calculating the heap sides that GSP-RM needs for booting.
> Although they technically represent distinct parts of the equations,
> they are always used together.
>
> A future change to this code will introduce runtime calculations based
> on values extracted from the GSP-RM, replacing the compile-time
> constants currently used. Merging the two types in this commit keeps
> the current functionality while providing a foundation that simplifies
> future changes.
>
> For the same reason, the LIBOSx constants are replaced with variables.
>
> Signed-off-by: Timur Tabi <ttabi@nvidia.com>
> ---
The direction of this series looks good to me, but I think there's some
unnecessary churn from moving the functions from LibosParams now, rather
than switching over after adding the TLV values to GspFwHeapParams. What
about doing essentially patch #1 then patch #3 (which doesn't need the
zero dummy vals if #2 is removed) of this series, then a patch each for
switching over
- non-wpr heap + pmu reserved size (+drop hal functions)
- vGPU stuff
- WPR2 heap size
I tried this locally and it seems to remove a bunch of the churn.
WDYT?
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 3/5] gpu: nova-core: read GSP heap variables from TLV
2026-11-10 23:10 [PATCH 0/5] Transition nova-core to use TLV variables for heap sizes Timur Tabi
2026-11-10 23:10 ` [PATCH 1/5] gpu: nova-core: add Tlv.get_u64 Timur Tabi
2026-11-10 23:10 ` [PATCH 2/5] gpu: nova-core: merge LibosParams into GspFwHeapParams Timur Tabi
@ 2026-11-10 23:10 ` Timur Tabi
2026-11-10 23:10 ` [PATCH 4/5] gpu: nova-core: use the TLV heap sizes Timur Tabi
` (2 subsequent siblings)
5 siblings, 0 replies; 13+ messages in thread
From: Timur Tabi @ 2026-11-10 23:10 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot, John Hubbard, Eliot Courtney,
nova-gpu
Update Nova-core to read and store the new heap size variables from
gsp.tlv. This change adds code that reads and stores the values and
also provides a constructor that uses them.
GSP-RM firmware TLVs (specifically gsp.tlv) that are based on r615 and
later contain an additional six variables that contain values used
to specify various heap sizes that GSP-RM needs in order to boot.
These values are specific to each GPU and RM version, and are obtained
by scraping them from the OpenRM source tree, just like every other
TLV variable.
Signed-off-by: Timur Tabi <ttabi@nvidia.com>
---
drivers/gpu/nova-core/firmware/tlv.rs | 1 -
drivers/gpu/nova-core/gsp/fw.rs | 56 ++++++++++++++++++++++++++-
2 files changed, 55 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/nova-core/firmware/tlv.rs b/drivers/gpu/nova-core/firmware/tlv.rs
index 75d8f907421b..0b79268c6dfb 100644
--- a/drivers/gpu/nova-core/firmware/tlv.rs
+++ b/drivers/gpu/nova-core/firmware/tlv.rs
@@ -258,7 +258,6 @@ pub(crate) fn load_file(
}
/// Return a u64, from either a 32-bit or 64-bit little-endian tag.
- #[expect(unused)]
pub(crate) fn get_u64(&self, tag: &[u8; 4]) -> Result<u64> {
let value = self.get_bytes(tag)?;
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index f30819fd89c2..f0edae9656d8 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -37,7 +37,8 @@
},
firmware::{
bindata::UcodesImage,
- gsp::GspFirmware, //
+ gsp::GspFirmware,
+ tlv::Tlv, //
},
gpu::{
Architecture,
@@ -74,6 +75,19 @@ pub(crate) struct GspFwHeapParams {
carveout_size: u64,
/// The minimum and maximum sizes allowed for the GSP FW heap, in bytes.
allowed_heap_size: Range<u64>,
+
+ #[expect(unused)]
+ tlv_gsp_fw_heap_base_size: u64,
+ #[expect(unused)]
+ tlv_gsp_hw_heap_size_per_gb: u64,
+ #[expect(unused)]
+ tlv_gsp_fw_heap_base_size_vgpu: u64,
+ #[expect(unused)]
+ tlv_gsp_fw_heap_size_per_vm: u64,
+ #[expect(unused)]
+ tlv_pmu_reserved_size: u32,
+ #[expect(unused)]
+ tlv_non_wpr_heap_size: u64,
}
/// Minimum required alignment for the GSP heap.
@@ -88,6 +102,13 @@ fn libos2() -> Self {
allowed_heap_size: cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MIN_MB => u64)
* u64::SZ_1M
..cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB => u64) * u64::SZ_1M,
+
+ tlv_gsp_fw_heap_base_size: 0,
+ tlv_gsp_hw_heap_size_per_gb: 0,
+ tlv_gsp_fw_heap_base_size_vgpu: 0,
+ tlv_gsp_fw_heap_size_per_vm: 0,
+ tlv_pmu_reserved_size: 0,
+ tlv_non_wpr_heap_size: 0,
}
}
@@ -101,6 +122,13 @@ fn libos3() -> Self {
* u64::SZ_1M
..cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MAX_MB => u64)
* u64::SZ_1M,
+
+ tlv_gsp_fw_heap_base_size: 0,
+ tlv_gsp_hw_heap_size_per_gb: 0,
+ tlv_gsp_fw_heap_base_size_vgpu: 0,
+ tlv_gsp_fw_heap_size_per_vm: 0,
+ tlv_pmu_reserved_size: 0,
+ tlv_non_wpr_heap_size: 0,
}
}
@@ -112,6 +140,13 @@ fn libos3_gh100() -> Self {
base_rm_size: cv!(bindings::GSP_FW_HEAP_PARAM_BASE_RM_SIZE_GH100),
carveout_size: base.carveout_size,
allowed_heap_size: base.allowed_heap_size,
+
+ tlv_gsp_fw_heap_base_size: 0,
+ tlv_gsp_hw_heap_size_per_gb: 0,
+ tlv_gsp_fw_heap_base_size_vgpu: 0,
+ tlv_gsp_fw_heap_size_per_vm: 0,
+ tlv_pmu_reserved_size: 0,
+ tlv_non_wpr_heap_size: 0,
}
}
@@ -127,6 +162,25 @@ pub(crate) fn from_chipset(chipset: Chipset) -> Self {
}
}
+ /// Returns the heap parameters from the TLV
+ #[expect(unused)]
+ pub(crate) fn from_tlv(tlv: &Tlv<'_>) -> Result<Self> {
+ Ok(Self {
+ // These three fields are not used for TLV-based implementations,
+ // so assign dummy values to them for now.
+ base_rm_size: 0,
+ carveout_size: 0,
+ allowed_heap_size: u64::MIN..u64::MAX,
+
+ tlv_gsp_fw_heap_base_size: tlv.get_u64(b"HBSZ")?,
+ tlv_gsp_hw_heap_size_per_gb: tlv.get_u64(b"HSPG")?,
+ tlv_gsp_fw_heap_base_size_vgpu: tlv.get_u64(b"HBSV")?,
+ tlv_gsp_fw_heap_size_per_vm: tlv.get_u64(b"HSPV")?,
+ tlv_pmu_reserved_size: tlv.get_u32(b"PRSZ")?,
+ tlv_non_wpr_heap_size: tlv.get_u64(b"GNWH")?,
+ })
+ }
+
/// Returns the WPR heap size to reserve when vGPU is enabled.
pub(crate) fn vgpu_wpr_heap_size(&self) -> u64 {
u64::from(bindings::GSP_FW_HEAP_SIZE_VGPU_DEFAULT)
--
2.54.0
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH 4/5] gpu: nova-core: use the TLV heap sizes
2026-11-10 23:10 [PATCH 0/5] Transition nova-core to use TLV variables for heap sizes Timur Tabi
` (2 preceding siblings ...)
2026-11-10 23:10 ` [PATCH 3/5] gpu: nova-core: read GSP heap variables from TLV Timur Tabi
@ 2026-11-10 23:10 ` Timur Tabi
2026-11-10 23:10 ` [PATCH 5/5] gpu: nova-core: remove heap size related HAL functions Timur Tabi
2026-11-10 23:21 ` [PATCH 0/5] Transition nova-core to use TLV variables for heap sizes Timur Tabi
5 siblings, 0 replies; 13+ messages in thread
From: Timur Tabi @ 2026-11-10 23:10 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot, John Hubbard, Eliot Courtney,
nova-gpu
Instead of calculating the various heap sizes needed by GSP-RM to boot
from constants and HAL functions, use the heap sizes read from gsp.tlv.
Signed-off-by: Timur Tabi <ttabi@nvidia.com>
---
drivers/gpu/nova-core/fb.rs | 31 ++++--
drivers/gpu/nova-core/fb/hal.rs | 2 +
drivers/gpu/nova-core/firmware/gsp.rs | 7 +-
drivers/gpu/nova-core/gsp/fw.rs | 143 ++++---------------------
drivers/gpu/nova-core/gsp/hal/gh100.rs | 2 +-
5 files changed, 47 insertions(+), 138 deletions(-)
diff --git a/drivers/gpu/nova-core/fb.rs b/drivers/gpu/nova-core/fb.rs
index 9a3314768385..ca1cb4af9ca5 100644
--- a/drivers/gpu/nova-core/fb.rs
+++ b/drivers/gpu/nova-core/fb.rs
@@ -242,7 +242,8 @@ pub(crate) fn new(
FbRange(fw_image_addr..fw_image_addr + fw_image_size)
};
- let (vf_partition_count, wpr2_heap_size) = wpr2_heap_params(chipset, vgpu_state, fb.end)?;
+ let (vf_partition_count, wpr2_heap_size) =
+ wpr2_heap_params(&gsp_fw.heap_params, vgpu_state, fb.end)?;
let wpr2_heap = {
const WPR2_HEAP_DOWN_ALIGN: Alignment = Alignment::new::<SZ_1M>();
@@ -264,7 +265,7 @@ pub(crate) fn new(
};
let non_wpr_heap = {
- let non_wpr_heap_size = hal.non_wpr_heap_size();
+ let non_wpr_heap_size = gsp_fw.heap_params.tlv_non_wpr_heap_size;
FbRange(wpr2.start - non_wpr_heap_size..wpr2.start)
};
@@ -278,7 +279,7 @@ pub(crate) fn new(
wpr2,
non_wpr_heap,
vf_partition_count,
- pmu_reserved_size: hal.pmu_reserved_size(),
+ pmu_reserved_size: gsp_fw.heap_params.tlv_pmu_reserved_size,
})
}
}
@@ -298,14 +299,16 @@ pub(crate) fn wpr2_range(bar: Bar0<'_>) -> Option<Range<u64>> {
}
/// Computes the number of VF partitions and the WPR2 heap size from the vGPU state.
-fn wpr2_heap_params(chipset: Chipset, vgpu_state: VgpuState, fb_size: u64) -> Result<(u8, u64)> {
- let heap_params = gsp::GspFwHeapParams::from_chipset(chipset);
-
+fn wpr2_heap_params(
+ heap_params: &gsp::GspFwHeapParams,
+ vgpu_state: VgpuState,
+ fb_size: u64,
+) -> Result<(u8, u64)> {
Ok(match vgpu_state {
VgpuState::Disabled => (0, heap_params.wpr_heap_size(fb_size)?),
VgpuState::Enabled { total_vfs } => (
u8::try_from(total_vfs.get()).map_err(|_| EINVAL)?,
- heap_params.vgpu_wpr_heap_size(),
+ heap_params.vgpu_wpr_heap_size(total_vfs.get())?,
),
})
}
@@ -327,16 +330,22 @@ pub(crate) struct FbSizes {
impl FbSizes {
/// Computes the framebuffer region sizes for GSP-FMC boot.
- pub(crate) fn new(chipset: Chipset, bar: Bar0<'_>, vgpu_state: VgpuState) -> Result<Self> {
+ pub(crate) fn new(
+ chipset: Chipset,
+ bar: Bar0<'_>,
+ gsp_fw: &GspFirmware<'_>,
+ vgpu_state: VgpuState,
+ ) -> Result<Self> {
let hal = hal::fb_hal(chipset);
let fb_size = hal.vidmem_size(bar);
- let (vf_partition_count, wpr2_heap_size) = wpr2_heap_params(chipset, vgpu_state, fb_size)?;
+ let (vf_partition_count, wpr2_heap_size) =
+ wpr2_heap_params(&gsp_fw.heap_params, vgpu_state, fb_size)?;
Ok(Self {
frts_size: hal.frts_size(),
wpr2_heap_size,
- non_wpr_heap_size: hal.non_wpr_heap_size(),
- pmu_reserved_size: hal.pmu_reserved_size(),
+ non_wpr_heap_size: gsp_fw.heap_params.tlv_non_wpr_heap_size,
+ pmu_reserved_size: gsp_fw.heap_params.tlv_pmu_reserved_size,
vf_partition_count,
})
}
diff --git a/drivers/gpu/nova-core/fb/hal.rs b/drivers/gpu/nova-core/fb/hal.rs
index 99363cfb116b..e30eb7de2a1c 100644
--- a/drivers/gpu/nova-core/fb/hal.rs
+++ b/drivers/gpu/nova-core/fb/hal.rs
@@ -34,9 +34,11 @@ pub(crate) trait FbHal {
fn vidmem_size(&self, bar: Bar0<'_>) -> u64;
/// Returns the amount of VRAM to reserve for the PMU.
+ #[expect(unused)]
fn pmu_reserved_size(&self) -> u32;
/// Returns the non-WPR heap size for this chipset, in bytes.
+ #[expect(unused)]
fn non_wpr_heap_size(&self) -> u64;
/// Returns the FRTS size, in bytes.
diff --git a/drivers/gpu/nova-core/firmware/gsp.rs b/drivers/gpu/nova-core/firmware/gsp.rs
index a2db7b6ba131..7fd518af9dba 100644
--- a/drivers/gpu/nova-core/firmware/gsp.rs
+++ b/drivers/gpu/nova-core/firmware/gsp.rs
@@ -18,7 +18,8 @@
Tlv,
},
},
- gpu::Chipset, //
+ gpu::Chipset,
+ gsp::GspFwHeapParams, //
};
/// The GSP firmware image, its signatures, and the GSP bootloader.
@@ -31,6 +32,8 @@ pub(crate) struct GspFirmware<'a> {
pub(crate) signatures: Coherent<'a, [u8]>,
/// GSP bootloader, verifies the GSP firmware before loading and running it.
pub(crate) bootloader: RiscvFirmware<'a>,
+ /// Heap sizing parameters reported by the firmware metadata.
+ pub(crate) heap_params: GspFwHeapParams,
}
impl<'a> GspFirmware<'a> {
@@ -48,6 +51,7 @@ pub(crate) fn new(
let fw_vvec = tlv.load_file(dev, chipset)?;
let signatures = Coherent::from_slice(dev, tlv.get_bytes(b"SIGN")?, GFP_KERNEL)?;
+ let heap_params = GspFwHeapParams::from_tlv(&tlv)?;
Ok(try_pin_init!(Self {
radix3 <- Radix3::new(dev, fw_vvec),
@@ -57,6 +61,7 @@ pub(crate) fn new(
RiscvFirmware::new(dev, &bl)?
},
+ heap_params,
}))
})
}
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index f0edae9656d8..8873282ac4cd 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -7,8 +7,6 @@
// Alias to avoid repeating the version number with every use.
use r000_00 as bindings;
-use core::ops::Range;
-
use kernel::{
bitfield,
dma::Coherent,
@@ -40,10 +38,6 @@
gsp::GspFirmware,
tlv::Tlv, //
},
- gpu::{
- Architecture,
- Chipset, //
- },
gsp::{
cmdq::Cmdq,
GSP_PAGE_SHIFT,
@@ -68,110 +62,23 @@
/// Heap memory requirements and constraints for booting GSP-RM.
pub(crate) struct GspFwHeapParams {
- /// The amount of heap memory used during GSP-RM boot and initialization (up to and including
- /// the first client subdevice allocation), in bytes.
- base_rm_size: u64,
- /// The base amount of heap required by the GSP operating system, in bytes.
- carveout_size: u64,
- /// The minimum and maximum sizes allowed for the GSP FW heap, in bytes.
- allowed_heap_size: Range<u64>,
-
- #[expect(unused)]
tlv_gsp_fw_heap_base_size: u64,
- #[expect(unused)]
tlv_gsp_hw_heap_size_per_gb: u64,
- #[expect(unused)]
tlv_gsp_fw_heap_base_size_vgpu: u64,
- #[expect(unused)]
tlv_gsp_fw_heap_size_per_vm: u64,
- #[expect(unused)]
- tlv_pmu_reserved_size: u32,
- #[expect(unused)]
- tlv_non_wpr_heap_size: u64,
+ pub(crate) tlv_pmu_reserved_size: u32,
+ pub(crate) tlv_non_wpr_heap_size: u64,
}
/// Minimum required alignment for the GSP heap.
const GSP_HEAP_ALIGNMENT: Alignment = Alignment::new::<{ 1 << 20 }>();
impl GspFwHeapParams {
- /// Version 2 of the GSP LIBOS (Turing and GA100)
- fn libos2() -> Self {
- Self {
- base_rm_size: cv!(bindings::GSP_FW_HEAP_PARAM_BASE_RM_SIZE_TU10X),
- carveout_size: cv!(bindings::GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS2),
- allowed_heap_size: cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MIN_MB => u64)
- * u64::SZ_1M
- ..cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB => u64) * u64::SZ_1M,
-
- tlv_gsp_fw_heap_base_size: 0,
- tlv_gsp_hw_heap_size_per_gb: 0,
- tlv_gsp_fw_heap_base_size_vgpu: 0,
- tlv_gsp_fw_heap_size_per_vm: 0,
- tlv_pmu_reserved_size: 0,
- tlv_non_wpr_heap_size: 0,
- }
- }
-
- /// Version 3 of the GSP LIBOS (GA102 - AD10x)
- fn libos3() -> Self {
- Self {
- base_rm_size: cv!(bindings::GSP_FW_HEAP_PARAM_BASE_RM_SIZE_TU10X),
- carveout_size: cv!(bindings::GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS3_BAREMETAL),
- allowed_heap_size: cv!(
- bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MIN_MB => u64)
- * u64::SZ_1M
- ..cv!(bindings::GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MAX_MB => u64)
- * u64::SZ_1M,
-
- tlv_gsp_fw_heap_base_size: 0,
- tlv_gsp_hw_heap_size_per_gb: 0,
- tlv_gsp_fw_heap_base_size_vgpu: 0,
- tlv_gsp_fw_heap_size_per_vm: 0,
- tlv_pmu_reserved_size: 0,
- tlv_non_wpr_heap_size: 0,
- }
- }
-
- /// Version 3 of the GSP LIBOS, (GH100+)
- fn libos3_gh100() -> Self {
- let base = Self::libos3();
-
- Self {
- base_rm_size: cv!(bindings::GSP_FW_HEAP_PARAM_BASE_RM_SIZE_GH100),
- carveout_size: base.carveout_size,
- allowed_heap_size: base.allowed_heap_size,
-
- tlv_gsp_fw_heap_base_size: 0,
- tlv_gsp_hw_heap_size_per_gb: 0,
- tlv_gsp_fw_heap_base_size_vgpu: 0,
- tlv_gsp_fw_heap_size_per_vm: 0,
- tlv_pmu_reserved_size: 0,
- tlv_non_wpr_heap_size: 0,
- }
- }
-
/// Returns the heap parameters corresponding to `chipset`.
///
/// The parameters reported by GSP-RM are initialized to zero.
- pub(crate) fn from_chipset(chipset: Chipset) -> Self {
- match chipset.arch() {
- Architecture::Turing => Self::libos2(),
- Architecture::Ampere if chipset < Chipset::GA102 => Self::libos2(),
- Architecture::Ampere | Architecture::Ada => Self::libos3(),
- _ => Self::libos3_gh100(),
- }
- }
-
- /// Returns the heap parameters from the TLV
- #[expect(unused)]
pub(crate) fn from_tlv(tlv: &Tlv<'_>) -> Result<Self> {
Ok(Self {
- // These three fields are not used for TLV-based implementations,
- // so assign dummy values to them for now.
- base_rm_size: 0,
- carveout_size: 0,
- allowed_heap_size: u64::MIN..u64::MAX,
-
tlv_gsp_fw_heap_base_size: tlv.get_u64(b"HBSZ")?,
tlv_gsp_hw_heap_size_per_gb: tlv.get_u64(b"HSPG")?,
tlv_gsp_fw_heap_base_size_vgpu: tlv.get_u64(b"HBSV")?,
@@ -182,44 +89,30 @@ pub(crate) fn from_tlv(tlv: &Tlv<'_>) -> Result<Self> {
}
/// Returns the WPR heap size to reserve when vGPU is enabled.
- pub(crate) fn vgpu_wpr_heap_size(&self) -> u64 {
- u64::from(bindings::GSP_FW_HEAP_SIZE_VGPU_DEFAULT)
- }
-
- /// Returns the amount of heap memory required to support a single channel allocation.
- fn client_alloc_size() -> u64 {
- u64::from(bindings::GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE)
+ pub(crate) fn vgpu_wpr_heap_size(&self, total_vfs: u16) -> Result<u64> {
+ //u64::from(bindings::GSP_FW_HEAP_SIZE_VGPU_DEFAULT)
+ let vm_size = self
+ .tlv_gsp_fw_heap_size_per_vm
+ .saturating_mul(u64::from(total_vfs))
.align_up(GSP_HEAP_ALIGNMENT)
- .unwrap_or(u64::MAX)
- }
+ .ok_or(EINVAL)?;
- /// Returns the amount of memory to reserve for management purposes for a framebuffer of size
- /// `fb_size`.
- fn management_overhead(fb_size: u64) -> Result<u64> {
- let fb_size_gb = fb_size.div_ceil(u64::SZ_1G);
-
- u64::from(bindings::GSP_FW_HEAP_PARAM_SIZE_PER_GB)
- .checked_mul(fb_size_gb)
- .ok_or(EINVAL)?
- .align_up(GSP_HEAP_ALIGNMENT)
- .ok_or(EINVAL)
+ Ok(self.tlv_gsp_fw_heap_base_size_vgpu.saturating_add(vm_size))
}
/// Returns the amount of memory (in bytes) to allocate for the WPR heap for a framebuffer size
/// of `fb_size` (in bytes).
pub(crate) fn wpr_heap_size(&self, fb_size: u64) -> Result<u64> {
- // The WPR heap will contain the following:
- // LIBOS carveout,
+ let fb_size_gb = fb_size.div_ceil(u64::SZ_1G);
+ let management_overhead: u64 = self
+ .tlv_gsp_hw_heap_size_per_gb
+ .saturating_mul(fb_size_gb)
+ .align_up(GSP_HEAP_ALIGNMENT)
+ .ok_or(EINVAL)?;
+
Ok(self
- .carveout_size
- // RM boot working memory,
- .saturating_add(self.base_rm_size)
- // One RM client,
- .saturating_add(Self::client_alloc_size())
- // Overhead for memory management.
- .saturating_add(Self::management_overhead(fb_size)?)
- // Clamp to the supported heap sizes.
- .clamp(self.allowed_heap_size.start, self.allowed_heap_size.end - 1))
+ .tlv_gsp_fw_heap_base_size
+ .saturating_add(management_overhead))
}
}
diff --git a/drivers/gpu/nova-core/gsp/hal/gh100.rs b/drivers/gpu/nova-core/gsp/hal/gh100.rs
index 91201b51030e..f43fbe3dd798 100644
--- a/drivers/gpu/nova-core/gsp/hal/gh100.rs
+++ b/drivers/gpu/nova-core/gsp/hal/gh100.rs
@@ -151,7 +151,7 @@ fn boot<'gpu>(
let chipset = ctx.chipset;
let gsp_falcon = ctx.gsp_falcon;
- let fb_sizes = FbSizes::new(chipset, ctx.bar, ctx.vgpu.state())?;
+ let fb_sizes = FbSizes::new(chipset, ctx.bar, gsp_fw, ctx.vgpu.state())?;
dev_dbg!(dev, "{:#x?}\n", fb_sizes);
let wpr_meta =
--
2.54.0
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH 5/5] gpu: nova-core: remove heap size related HAL functions
2026-11-10 23:10 [PATCH 0/5] Transition nova-core to use TLV variables for heap sizes Timur Tabi
` (3 preceding siblings ...)
2026-11-10 23:10 ` [PATCH 4/5] gpu: nova-core: use the TLV heap sizes Timur Tabi
@ 2026-11-10 23:10 ` Timur Tabi
2026-11-10 23:21 ` [PATCH 0/5] Transition nova-core to use TLV variables for heap sizes Timur Tabi
5 siblings, 0 replies; 13+ messages in thread
From: Timur Tabi @ 2026-11-10 23:10 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot, John Hubbard, Eliot Courtney,
nova-gpu
Now that Nova-core reads all heap size values from the TLVs, we can
remove all of the HAL functions that used to provide that data.
Signed-off-by: Timur Tabi <ttabi@nvidia.com>
---
drivers/gpu/nova-core/fb/hal.rs | 8 ------
drivers/gpu/nova-core/fb/hal/ga100.rs | 8 ------
drivers/gpu/nova-core/fb/hal/ga102.rs | 8 ------
drivers/gpu/nova-core/fb/hal/gb100.rs | 41 +--------------------------
drivers/gpu/nova-core/fb/hal/gb202.rs | 13 +--------
drivers/gpu/nova-core/fb/hal/gh100.rs | 12 +-------
drivers/gpu/nova-core/fb/hal/tu102.rs | 16 -----------
7 files changed, 3 insertions(+), 103 deletions(-)
diff --git a/drivers/gpu/nova-core/fb/hal.rs b/drivers/gpu/nova-core/fb/hal.rs
index e30eb7de2a1c..53821e4a70dd 100644
--- a/drivers/gpu/nova-core/fb/hal.rs
+++ b/drivers/gpu/nova-core/fb/hal.rs
@@ -33,14 +33,6 @@ pub(crate) trait FbHal {
/// Returns the VRAM size, in bytes.
fn vidmem_size(&self, bar: Bar0<'_>) -> u64;
- /// Returns the amount of VRAM to reserve for the PMU.
- #[expect(unused)]
- fn pmu_reserved_size(&self) -> u32;
-
- /// Returns the non-WPR heap size for this chipset, in bytes.
- #[expect(unused)]
- fn non_wpr_heap_size(&self) -> u64;
-
/// Returns the FRTS size, in bytes.
fn frts_size(&self) -> u64;
}
diff --git a/drivers/gpu/nova-core/fb/hal/ga100.rs b/drivers/gpu/nova-core/fb/hal/ga100.rs
index a81b3e42ae41..d0950d1f2339 100644
--- a/drivers/gpu/nova-core/fb/hal/ga100.rs
+++ b/drivers/gpu/nova-core/fb/hal/ga100.rs
@@ -70,14 +70,6 @@ fn vidmem_size(&self, bar: Bar0<'_>) -> u64 {
super::tu102::vidmem_size_gp102(bar)
}
- fn pmu_reserved_size(&self) -> u32 {
- super::tu102::pmu_reserved_size_tu102()
- }
-
- fn non_wpr_heap_size(&self) -> u64 {
- super::tu102::non_wpr_heap_size_tu102()
- }
-
// GA100 is a special case where its FRTS region exists, but is empty. We
// return a size of 0 because we still need to record where the region starts.
fn frts_size(&self) -> u64 {
diff --git a/drivers/gpu/nova-core/fb/hal/ga102.rs b/drivers/gpu/nova-core/fb/hal/ga102.rs
index a5eeddb8a1ae..837bac1b7b2c 100644
--- a/drivers/gpu/nova-core/fb/hal/ga102.rs
+++ b/drivers/gpu/nova-core/fb/hal/ga102.rs
@@ -37,14 +37,6 @@ fn vidmem_size(&self, bar: Bar0<'_>) -> u64 {
vidmem_size_ga102(bar)
}
- fn pmu_reserved_size(&self) -> u32 {
- super::tu102::pmu_reserved_size_tu102()
- }
-
- fn non_wpr_heap_size(&self) -> u64 {
- super::tu102::non_wpr_heap_size_tu102()
- }
-
fn frts_size(&self) -> u64 {
super::tu102::frts_size_tu102()
}
diff --git a/drivers/gpu/nova-core/fb/hal/gb100.rs b/drivers/gpu/nova-core/fb/hal/gb100.rs
index 7e6dc40dff32..53d403d9487b 100644
--- a/drivers/gpu/nova-core/fb/hal/gb100.rs
+++ b/drivers/gpu/nova-core/fb/hal/gb100.rs
@@ -11,12 +11,7 @@
Mmio, //
},
num::Bounded,
- prelude::*,
- ptr::{
- const_align_up,
- Alignment, //
- },
- sizes::*, //
+ prelude::*, //
};
use crate::{
@@ -78,31 +73,6 @@ fn write_sysmem_flush_page_gb100(hshub0: Mmio<'_, regs::Hshub0Registers>, addr:
hshub0.write_reg(regs::NV_PFB_HSHUB_EG_PCIE_FLUSH_SYSMEM_ADDR_LO::zeroed().with_adr(addr_lo));
}
-/// PMU backing store, `kpmuReservedMemoryBackingStoreSizeGet` in OpenRM for the chipsets whose
-/// PMU initializes itself: 9 MiB + 320 KiB.
-const PMU_BACKING_STORE_SIZE: usize = 0x95_0000;
-
-/// PMU surfaces, `kpmuReservedMemorySurfacesSizeGet` in OpenRM.
-const PMU_SURFACES_SIZE: usize = SZ_16M + SZ_256K;
-
-/// Miscellaneous PMU memory.
-const PMU_MISC_SIZE: usize = SZ_4K;
-
-/// Alignment of the PMU reserved region, `KPMU_RESERVED_MEMORY_ALIGNMENT` in OpenRM.
-const PMU_RESERVED_MEMORY_ALIGNMENT: Alignment = Alignment::new::<SZ_128K>();
-
-/// PMU region above FRTS: the backing store, the surfaces and the miscellaneous memory, aligned to
-/// [`PMU_RESERVED_MEMORY_ALIGNMENT`].
-const PMU_RESERVED_SIZE: usize = const_align_up(
- PMU_BACKING_STORE_SIZE + PMU_SURFACES_SIZE + PMU_MISC_SIZE,
- PMU_RESERVED_MEMORY_ALIGNMENT,
-)
-.unwrap();
-
-pub(super) const fn pmu_reserved_size_gb100() -> u32 {
- cv!(PMU_RESERVED_SIZE)
-}
-
impl FbHal for Gb100 {
fn read_sysmem_flush_page(&self, bar: Bar0<'_>) -> u64 {
read_sysmem_flush_page_gb100(hshub0(bar))
@@ -124,15 +94,6 @@ fn vidmem_size(&self, bar: Bar0<'_>) -> u64 {
super::ga102::vidmem_size_ga102(bar)
}
- fn pmu_reserved_size(&self) -> u32 {
- pmu_reserved_size_gb100()
- }
-
- fn non_wpr_heap_size(&self) -> u64 {
- // Non-WPR heap for GB10x (see Open RM: kgspGetNonWprHeapSize, GB100/GB102).
- u64::SZ_2M
- }
-
fn frts_size(&self) -> u64 {
super::tu102::frts_size_tu102()
}
diff --git a/drivers/gpu/nova-core/fb/hal/gb202.rs b/drivers/gpu/nova-core/fb/hal/gb202.rs
index 1d36954a2776..90673ef4a05a 100644
--- a/drivers/gpu/nova-core/fb/hal/gb202.rs
+++ b/drivers/gpu/nova-core/fb/hal/gb202.rs
@@ -6,8 +6,7 @@
use kernel::{
io::Io,
num::Bounded,
- prelude::*,
- sizes::SizeConstants, //
+ prelude::*, //
};
use crate::{
@@ -67,16 +66,6 @@ fn vidmem_size(&self, bar: Bar0<'_>) -> u64 {
super::ga102::vidmem_size_ga102(bar)
}
- fn pmu_reserved_size(&self) -> u32 {
- super::gb100::pmu_reserved_size_gb100()
- }
-
- fn non_wpr_heap_size(&self) -> u64 {
- // Non-WPR heap for GB20x (see OpenRM: kgspGetNonWprHeapSize, GB202+). This size is
- // r000-specific.
- 3 * u64::SZ_1M
- }
-
fn frts_size(&self) -> u64 {
super::tu102::frts_size_tu102()
}
diff --git a/drivers/gpu/nova-core/fb/hal/gh100.rs b/drivers/gpu/nova-core/fb/hal/gh100.rs
index cf8e6403ef98..945760953ee4 100644
--- a/drivers/gpu/nova-core/fb/hal/gh100.rs
+++ b/drivers/gpu/nova-core/fb/hal/gh100.rs
@@ -4,8 +4,7 @@
use kernel::{
io::Io,
num::Bounded,
- prelude::*,
- sizes::SizeConstants, //
+ prelude::*, //
};
use crate::{
@@ -59,15 +58,6 @@ fn vidmem_size(&self, bar: Bar0<'_>) -> u64 {
super::ga102::vidmem_size_ga102(bar)
}
- fn pmu_reserved_size(&self) -> u32 {
- super::tu102::pmu_reserved_size_tu102()
- }
-
- fn non_wpr_heap_size(&self) -> u64 {
- // Non-WPR heap for Hopper (see Open RM: kgspCalculateFbLayout_GH100).
- u64::SZ_2M
- }
-
fn frts_size(&self) -> u64 {
super::tu102::frts_size_tu102()
}
diff --git a/drivers/gpu/nova-core/fb/hal/tu102.rs b/drivers/gpu/nova-core/fb/hal/tu102.rs
index bf5967cbffd3..fac30be446c7 100644
--- a/drivers/gpu/nova-core/fb/hal/tu102.rs
+++ b/drivers/gpu/nova-core/fb/hal/tu102.rs
@@ -42,14 +42,6 @@ pub(super) fn vidmem_size_gp102(bar: Bar0<'_>) -> u64 {
.usable_fb_size()
}
-pub(super) const fn pmu_reserved_size_tu102() -> u32 {
- 0
-}
-
-pub(super) const fn non_wpr_heap_size_tu102() -> u64 {
- u64::SZ_1M
-}
-
pub(super) const fn frts_size_tu102() -> u64 {
u64::SZ_1M
}
@@ -73,14 +65,6 @@ fn vidmem_size(&self, bar: Bar0<'_>) -> u64 {
vidmem_size_gp102(bar)
}
- fn pmu_reserved_size(&self) -> u32 {
- pmu_reserved_size_tu102()
- }
-
- fn non_wpr_heap_size(&self) -> u64 {
- non_wpr_heap_size_tu102()
- }
-
fn frts_size(&self) -> u64 {
frts_size_tu102()
}
--
2.54.0
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH 0/5] Transition nova-core to use TLV variables for heap sizes
2026-11-10 23:10 [PATCH 0/5] Transition nova-core to use TLV variables for heap sizes Timur Tabi
` (4 preceding siblings ...)
2026-11-10 23:10 ` [PATCH 5/5] gpu: nova-core: remove heap size related HAL functions Timur Tabi
@ 2026-11-10 23:21 ` Timur Tabi
5 siblings, 0 replies; 13+ messages in thread
From: Timur Tabi @ 2026-11-10 23:21 UTC (permalink / raw)
To: nova-gpu@lists.linux.dev, Alexandre Courbot, dakr@kernel.org,
Eliot Courtney, John Hubbard
On Tue, 2026-11-10 at 17:10 -0600, Timur Tabi wrote:
> Starting with r615, the GSP-RM firmware TLVs (i.e. gsp.tlv) contain an
> an additional six variables that that should be used to calculate the
> various heap sizes that GSP-RM needs in order to boot.
>
> These values are specific to each GPU and RM version, and are obtained
> by scraping them from the OpenRM source tree.
I forgot to mention that this is based on Alex's r000_rebase_all branch:
https://github.com/Gnurou/linux/tree/r000_rebase_all
^ permalink raw reply [flat|nested] 13+ messages in thread