NVIDIA GPU driver infrastructure
 help / color / mirror / Atom feed
From: "Eliot Courtney" <ecourtney@nvidia.com>
To: "Alexandre Courbot" <acourbot@nvidia.com>,
	"Danilo Krummrich" <dakr@kernel.org>,
	"John Hubbard" <jhubbard@nvidia.com>
Cc: "Timur Tabi" <ttabi@nvidia.com>,
	"Alistair Popple" <apopple@nvidia.com>,
	"Eliot Courtney" <ecourtney@nvidia.com>,
	"Shashank Sharma" <shashanks@nvidia.com>,
	"Zhi Wang" <zhiw@nvidia.com>, "David Airlie" <airlied@gmail.com>,
	"Simona Vetter" <simona@ffwll.ch>,
	"Bjorn Helgaas" <bhelgaas@google.com>,
	"Miguel Ojeda" <ojeda@kernel.org>, "Gary Guo" <gary@garyguo.net>,
	"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
	"Benno Lossin" <lossin@kernel.org>,
	"Andreas Hindborg" <a.hindborg@kernel.org>,
	"Alice Ryhl" <aliceryhl@google.com>,
	"Trevor Gross" <tmgross@umich.edu>,
	nova-gpu@lists.linux.dev, LKML <linux-kernel@vger.kernel.org>,
	"Boqun Feng" <boqun@kernel.org>
Subject: Re: [PATCH v13 7/9] gpu: nova-core: Hopper/Blackwell: add GSP lockdown release polling
Date: Wed, 03 Jun 2026 20:53:41 +0900	[thread overview]
Message-ID: <DIZE6PDLOCHP.ITQSKNYRZ7H1@nvidia.com> (raw)
In-Reply-To: <DIZC53CCKAUY.2L04G0C2WSVQZ@nvidia.com>

On Wed Jun 3, 2026 at 7:17 PM JST, Alexandre Courbot wrote:
> On Wed Jun 3, 2026 at 4:30 PM JST, Alexandre Courbot wrote:
>> From: John Hubbard <jhubbard@nvidia.com>
>>
>> On Hopper and Blackwell, FSP boots GSP with hardware lockdown enabled.
>> After FSP Chain of Trust completes, the driver must poll for lockdown
>> release before proceeding with GSP initialization. Add the register
>> bit and helper functions needed for this polling.
>>
>> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
>> Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
>> ---
>>  drivers/gpu/nova-core/falcon/gsp.rs    |  6 +++
>>  drivers/gpu/nova-core/fsp.rs           |  6 +++
>>  drivers/gpu/nova-core/gsp/hal/gh100.rs | 88 +++++++++++++++++++++++++++++++++-
>>  drivers/gpu/nova-core/regs.rs          |  2 +
>>  4 files changed, 100 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpu/nova-core/falcon/gsp.rs b/drivers/gpu/nova-core/falcon/gsp.rs
>> index df6d5a382c7a..136d6b24103f 100644
>> --- a/drivers/gpu/nova-core/falcon/gsp.rs
>> +++ b/drivers/gpu/nova-core/falcon/gsp.rs
>> @@ -57,4 +57,10 @@ pub(crate) fn check_reload_completed(&self, bar: &Bar0, timeout: Delta) -> Resul
>>          )
>>          .map(|_| true)
>>      }
>> +
>> +    /// Returns whether the RISC-V branch privilege lockdown bit is set.
>> +    pub(crate) fn riscv_branch_privilege_lockdown(&self, bar: &Bar0) -> bool {
>> +        bar.read(regs::NV_PFALCON_FALCON_HWCFG2::of::<Gsp>())
>> +            .riscv_br_priv_lockdown()
>> +    }
>>  }
>> diff --git a/drivers/gpu/nova-core/fsp.rs b/drivers/gpu/nova-core/fsp.rs
>> index 883ac4f8b811..872898ffe0a3 100644
>> --- a/drivers/gpu/nova-core/fsp.rs
>> +++ b/drivers/gpu/nova-core/fsp.rs
>> @@ -184,6 +184,12 @@ pub(crate) fn new(
>>              resume,
>>          })
>>      }
>> +
>> +    /// DMA address of the FMC boot parameters, needed after boot for lockdown
>> +    /// release polling.
>> +    pub(crate) fn boot_params_dma_handle(&self) -> u64 {
>> +        self.fmc_boot_params.dma_handle()
>> +    }
>>  }
>>  
>>  /// FSP interface for Hopper/Blackwell GPUs.
>> diff --git a/drivers/gpu/nova-core/gsp/hal/gh100.rs b/drivers/gpu/nova-core/gsp/hal/gh100.rs
>> index f41f3fea15ff..def41745a30f 100644
>> --- a/drivers/gpu/nova-core/gsp/hal/gh100.rs
>> +++ b/drivers/gpu/nova-core/gsp/hal/gh100.rs
>> @@ -5,7 +5,9 @@
>>  
>>  use kernel::{
>>      device,
>> -    dma::Coherent, //
>> +    dma::Coherent,
>> +    io::poll::read_poll_timeout,
>> +    time::Delta, //
>>  };
>>  
>>  use crate::{
>> @@ -33,6 +35,86 @@
>>      },
>>  };
>>  
>> +/// GSP lockdown pattern written by firmware to mbox0 while RISC-V branch privilege
>> +/// lockdown is active. The low byte varies, the upper 24 bits are fixed.
>> +const GSP_LOCKDOWN_PATTERN: u32 = 0xbadf_4100;
>> +const GSP_LOCKDOWN_MASK: u32 = 0xffff_ff00;

nit: these constants can be moved into impl GspMbox or made local to
`is_locked_down`

>> +
>> +/// GSP falcon mailbox state, used to track lockdown release status.
>> +struct GspMbox {
>> +    mbox0: u32,
>> +    mbox1: u32,
>> +}
>> +
>> +impl GspMbox {
>> +    /// Reads both mailboxes from the GSP falcon.
>> +    fn read(gsp_falcon: &Falcon<GspEngine>, bar: &Bar0) -> Self {
>> +        Self {
>> +            mbox0: gsp_falcon.read_mailbox0(bar),
>> +            mbox1: gsp_falcon.read_mailbox1(bar),
>> +        }
>> +    }
>> +
>> +    /// Returns `true` if the lockdown pattern is present in `mbox0`.
>> +    fn is_locked_down(&self) -> bool {
>> +        (self.mbox0 & GSP_LOCKDOWN_MASK) == GSP_LOCKDOWN_PATTERN
>> +    }
>> +
>> +    /// Combines mailbox0 and mailbox1 into a 64-bit address.
>> +    fn combined_addr(&self) -> u64 {
>> +        (u64::from(self.mbox1) << 32) | u64::from(self.mbox0)
>> +    }
>> +
>> +    /// Returns `true` if GSP lockdown has been released.
>> +    ///
>> +    /// Checks the lockdown pattern, validates the boot params address,
>> +    /// and verifies the `HWCFG2` lockdown bit is clear.
>> +    fn lockdown_released(
>> +        &self,
>> +        gsp_falcon: &Falcon<GspEngine>,
>> +        bar: &Bar0,
>> +        fmc_boot_params_addr: u64,
>> +    ) -> bool {
>> +        if self.is_locked_down() {
>> +            return false;
>> +        }
>> +
>> +        if self.mbox0 != 0 && self.combined_addr() != fmc_boot_params_addr {
>> +            return true;
>> +        }
>
> This looks like a bug - if the mailboxes still contain the boot
> parameters address, we will keep going and might return true on the next
> line, which the caller will interpret as an error. OpenRM does the
> opposite check and has an additional test for `mailbox0 != 0`, which we
> can translate into this logic:
>
>     if self.mbox0 != 0 {
>         return self.combined_addr() != fmc_boot_params_addr;
>     }
>
> I'll fix it and add a few comments explaining what the code does as it
> can be a bit convoluted.

Yeah, I agree. I think the logic openrm uses is like:

1. wait until HWCFG2 != 0 && (HWCFG2 & 0xffffff00) != 0xbadf4100
2. wait until mbox0 == 0 || addr != fmc_boot_params_addr
3. wait until riscv_br_priv_lockdown == 0 || mbox0 != 0

So several things are different to openrm (not sure if they are wrong
though):

1. `is_locked_down` is checking the wrong register (should check HWCFG2
   AFAICT). I think we should name this more like
   'gsp_mailboxes_readable' or something, since IIUC it is meant to test
   when it's valid to read mboxs.
2. other logic is wrong as you mentioned.

Maybe we could structure lockdown_released like this:

// can't read the mailboxes yet (even though we already did but the
// result is actually not valid so maybe this should be restructured)
if !gsp_mailboxes_readable {
  return false;
}

// we can read the registers and we are still waiting for mbox0 to
// be zero
if mbox0 != 0 && addr == fmc_boot_params_addr {
  return false;
}

// either lockdown is released or some error happened
return riscv_br_priv_lockdown == 0 || mbox0 != 0

I think your suggested change w.r.t. if self.mbox0 != 0; is also
correct. I think we should update the comment on the function and the
function name too to say it returns true on lockdown release OR an error
happened.

I don't know if there's a simpler logic that will work, just commenting
on how it compares to openrm.

  reply	other threads:[~2026-06-03 11:53 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-03  7:30 [PATCH v13 0/9] gpu: nova-core: Hopper/Blackwell support Alexandre Courbot
2026-06-03  7:30 ` [PATCH v13 1/9] gpu: nova-core: Hopper/Blackwell: add FSP falcon EMEM operations Alexandre Courbot
2026-06-03 10:49   ` Eliot Courtney
2026-06-03  7:30 ` [PATCH v13 2/9] gpu: nova-core: Hopper/Blackwell: add FSP message infrastructure Alexandre Courbot
2026-06-03 10:50   ` Eliot Courtney
2026-06-03  7:30 ` [PATCH v13 3/9] gpu: nova-core: add MCTP/NVDM protocol types for firmware communication Alexandre Courbot
2026-06-03  7:30 ` [PATCH v13 4/9] gpu: nova-core: Hopper/Blackwell: add FSP send/receive messaging Alexandre Courbot
2026-06-03 10:57   ` Eliot Courtney
2026-06-03  7:30 ` [PATCH v13 5/9] gpu: nova-core: Hopper/Blackwell: select FSP Chain of Trust version Alexandre Courbot
2026-06-03  7:30 ` [PATCH v13 6/9] gpu: nova-core: Hopper/Blackwell: add FSP Chain of Trust boot Alexandre Courbot
2026-06-03 12:47   ` Eliot Courtney
2026-06-03 13:35     ` Alexandre Courbot
2026-06-03  7:30 ` [PATCH v13 7/9] gpu: nova-core: Hopper/Blackwell: add GSP lockdown release polling Alexandre Courbot
2026-06-03 10:17   ` Alexandre Courbot
2026-06-03 11:53     ` Eliot Courtney [this message]
2026-06-03 14:53       ` Alexandre Courbot
2026-06-03  7:30 ` [PATCH v13 8/9] gpu: nova-core: add non-sec2 unload path Alexandre Courbot
2026-06-03  7:30 ` [PATCH v13 9/9] gpu: nova-core: gsp: enable FSP boot path Alexandre Courbot
2026-06-03 11:04   ` Eliot Courtney
2026-06-03 15:04 ` [PATCH v13 0/9] gpu: nova-core: Hopper/Blackwell support Alexandre Courbot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DIZE6PDLOCHP.ITQSKNYRZ7H1@nvidia.com \
    --to=ecourtney@nvidia.com \
    --cc=a.hindborg@kernel.org \
    --cc=acourbot@nvidia.com \
    --cc=airlied@gmail.com \
    --cc=aliceryhl@google.com \
    --cc=apopple@nvidia.com \
    --cc=bhelgaas@google.com \
    --cc=bjorn3_gh@protonmail.com \
    --cc=boqun@kernel.org \
    --cc=dakr@kernel.org \
    --cc=gary@garyguo.net \
    --cc=jhubbard@nvidia.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lossin@kernel.org \
    --cc=nova-gpu@lists.linux.dev \
    --cc=ojeda@kernel.org \
    --cc=shashanks@nvidia.com \
    --cc=simona@ffwll.ch \
    --cc=tmgross@umich.edu \
    --cc=ttabi@nvidia.com \
    --cc=zhiw@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox