From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E5E92D5924 for ; Sun, 7 Dec 2025 08:31:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.53 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765096267; cv=none; b=dNkrIN3pfvkBQzNhWqaDIANsiJX5PJFihHzX/OcyEb4oGhkBy6ORjEyGVLjsaSBDT+J8MlFYxk9U6s4ktHp+SJ2rLKY1RdZWuFlH6JssvWCtvanNzXSo3ULX9GUoCxUTe3SKyZZMdmMjwdIjM1hhYSurciG054FNPrBRl8rn6pk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765096267; c=relaxed/simple; bh=/LR1XGcTPw/8s4MqENvx9JIq/PEmCHmXFp+zwo/6N0o=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=vFUQ0U9C/Z8VGgK2b9fTTOS5P2/V2rR3Dy1EPDOhnSA0s7GD+wIrv3+M9IZZt8RNNElwbGh0JiPCJ/MO5CLHpqMet/etfbUPcNnCXK1SE8rgf+TGpE2fpwC5l+09qKweFy/asNU4bVHivusZK5WInj/2Z5XH3TYBaQEsEGxpTME= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=aaRFZI2R; arc=none smtp.client-ip=209.85.208.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aaRFZI2R" Received: by mail-ed1-f53.google.com with SMTP id 4fb4d7f45d1cf-6492e7925d2so309299a12.3 for ; Sun, 07 Dec 2025 00:31:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765096263; x=1765701063; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=wUAqt5njeLVh6MrbdJefpUGQCUq+3iNdkFktE67ektk=; b=aaRFZI2RAmY3zaIfSnHAde5Cvi9nj9gIaqP+veyh/QOHQ9ashlyNuXaY56ou6GWix3 Es1HsVwJG1LbsCw6Cu9YBFSQ6AHnyeA+nA5DrSAQwlZ4RzExlq6GTP8xosLF+yiqy7mc onl9aibmpD41tJ2A9X6xqliixUwugroywLax28nLju7+nxrr7EONcLzNs+sbo0By42fc wmQ0SG2jqqIhRsppC0a4m/RrqaSb30DIEnPxdcm8DBF/+KaoB5n+yDpOtCeDLWuEWrrM ZoZwI8wNVgUVys3E9lv0dGsyDeBwvN/c+cGgqIuZE61Wo4JMeJiYLvAeysZWA/S65em9 etww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765096263; x=1765701063; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=wUAqt5njeLVh6MrbdJefpUGQCUq+3iNdkFktE67ektk=; b=IpgLNjSx5k/CioUX5OYcsMlRpS1KQS/zevvtOXBYFlK+0dqMcQnzHCL7uCd0ua9Vay dqiNvu/pHGOBoAyNjQ2uj2kntz6Ycixp7Frs6oN+nWgoPHFSsAN9u87rjeNNsx9YfaFp tvYuKmObw7V1MkM821O0b226JHlKnp56gkfzB0JYYr4n6GH+Z1Q74p4RI1hMM1DQ+qoL eL7wJpF5ebNT4ouDhszfO6Viapx3s1kx5JA49A3qJYPdZpDTCqdnQ9x+L45wkIAvMJxT 5GOmKPSTDz/joIPKSkyYETRIY7BV1WCRCigeYSVskpvQea0ukpEY8P8lluZ/abSZBFma wD2A== X-Forwarded-Encrypted: i=1; AJvYcCU5P81sjaYkdKwRGBTdbjVY47PXGs2byAqVVjBDh64lL3RJ/BmZQbIUG0QC6Zm0QaL4jVoc/2a7Z124NgE=@vger.kernel.org X-Gm-Message-State: AOJu0YwJS02XrLhYQ1EpFXNSFf5iH25zRQ7fLgy4Mh9dcgCYUYrzNkM2 IgPiRSCcYj7VhPACF92Lb+EhJEIC4w9JMvqkJpmlcz1d83HGWhUaXpbaeo0oHg== X-Gm-Gg: ASbGncvJ9deDyIQ5jfHPaqvmCZ3jU+qjJ4waQVqy3wLYR2jXtPqAkUtMvgrfksW4t/J MuCJPLlzW9YuTuy4vZae/9QIS5q/nCvHeQvJVN8HNJeusMSd08BxDShpBJVfkFV1+XiKv5Tcfxn 7xjbsCQ8Z/2jbcJQWfHeutzlJzND1e0yJElGYfX0UwPhq52ZikxMMjKEc07gf9Ok6eGNloUvwMT 47twkFOjRByvl9K4ktOOXiX++qe7tEgNnVJsb+ufZgQ/w8IqceuOlmkTSNodZG8qvkCeNP+T9+s OG1Yh9lGNOqNyXAkhK3yrfXjzuNieKpb0tcyG9n40lYFxSfBnnc9jJ9LuQapHSUYjw53GmA+n1h HgUcwcQzIyTXXazxx3EUonqc4Y5+NwL4VZsQaS7YD33x1YBZRMP+eRpqawbPt3xutDkUF4pp0ux l4S/lPtSRmlQRFDYKC4RPVonT2oVx3PicvTErwO5quVgCAHWKUJoMW/SvlYyCm5frEL6jXXYm2j B6bJv9qSTo+kMoFbnZb0dCiLJoUWVHWJ3YWRg+uREQ= X-Google-Smtp-Source: AGHT+IEVqNfD63Og/iUd5redAtxSFIwiSLJ5eCxMwzhmCuXyLMf5his7H3QmVu2l6AnzMWhPzE8plg== X-Received: by 2002:a05:600c:3b07:b0:477:1af2:f40a with SMTP id 5b1f17b1804b1-47939e242f1mr45974225e9.17.1765089776787; Sat, 06 Dec 2025 22:42:56 -0800 (PST) Received: from ?IPV6:2003:df:bf22:3c00:1c42:64e:ef2a:93cd? (p200300dfbf223c001c42064eef2a93cd.dip0.t-ipconnect.de. [2003:df:bf22:3c00:1c42:64e:ef2a:93cd]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47930ca15adsm179162575e9.13.2025.12.06.22.42.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 06 Dec 2025 22:42:56 -0800 (PST) Message-ID: <7fe8b05d-cb49-48bd-ac0a-d993e173924c@gmail.com> Date: Sun, 7 Dec 2025 07:42:54 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC 7/7] gpu: nova-core: load the scrubber ucode when vGPU support is enabled To: Zhi Wang , rust-for-linux@vger.kernel.org, linux-pci@vger.kernel.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: airlied@gmail.com, dakr@kernel.org, aliceryhl@google.com, bhelgaas@google.com, kwilczynski@kernel.org, ojeda@kernel.org, alex.gaynor@gmail.com, boqun.feng@gmail.com, gary@garyguo.net, bjorn3_gh@protonmail.com, lossin@kernel.org, a.hindborg@kernel.org, tmgross@umich.edu, markus.probst@posteo.de, helgaas@kernel.org, cjia@nvidia.com, alex@shazbot.org, smitra@nvidia.com, ankita@nvidia.com, aniketa@nvidia.com, kwankhede@nvidia.com, targupta@nvidia.com, acourbot@nvidia.com, joelagnelf@nvidia.com, jhubbard@nvidia.com, zhiwang@kernel.org References: <20251206124208.305963-1-zhiw@nvidia.com> <20251206124208.305963-8-zhiw@nvidia.com> Content-Language: de-AT-frami From: Dirk Behme In-Reply-To: <20251206124208.305963-8-zhiw@nvidia.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 06.12.25 13:42, Zhi Wang wrote: > To support the maximum vGPUs on the device that support vGPU, a larger > WPR2 heap size is required. By setting the WPR2 heap size larger than 256MB > the scrubber ucode image is required to scrub the FB memory before any > other ucode image is executed. > > If not, the GSP firmware hangs when booting. > > When vGPU support is enabled, execute the scrubber ucode image to scrub the > FB memory before executing any other ucode images. > > Signed-off-by: Zhi Wang > --- > drivers/gpu/nova-core/firmware.rs | 1 + > drivers/gpu/nova-core/firmware/booter.rs | 2 ++ > drivers/gpu/nova-core/gsp/boot.rs | 27 ++++++++++++++++++++++++ > drivers/gpu/nova-core/regs.rs | 11 ++++++++++ > 4 files changed, 41 insertions(+) > > diff --git a/drivers/gpu/nova-core/firmware.rs b/drivers/gpu/nova-core/firmware.rs > index 2d2008b33fb4..5ae1ab262d57 100644 > --- a/drivers/gpu/nova-core/firmware.rs > +++ b/drivers/gpu/nova-core/firmware.rs > @@ -226,6 +226,7 @@ const fn make_entry_chipset(self, chipset: &str) -> Self { > .make_entry_file(chipset, "booter_unload") > .make_entry_file(chipset, "bootloader") > .make_entry_file(chipset, "gsp") > + .make_entry_file(chipset, "scrubber") > } > > pub(crate) const fn create( > diff --git a/drivers/gpu/nova-core/firmware/booter.rs b/drivers/gpu/nova-core/firmware/booter.rs > index f107f753214a..f622c9b960de 100644 > --- a/drivers/gpu/nova-core/firmware/booter.rs > +++ b/drivers/gpu/nova-core/firmware/booter.rs > @@ -269,6 +269,7 @@ fn new_booter(dev: &device::Device, data: &[u8]) -> Result > > #[derive(Copy, Clone, Debug, PartialEq)] > pub(crate) enum BooterKind { > + Scrubber, > Loader, > #[expect(unused)] > Unloader, > @@ -286,6 +287,7 @@ pub(crate) fn new( > bar: &Bar0, > ) -> Result { > let fw_name = match kind { > + BooterKind::Scrubber => "scrubber", > BooterKind::Loader => "booter_load", > BooterKind::Unloader => "booter_unload", > }; > diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs > index ec006c26f19f..8ef79433f017 100644 > --- a/drivers/gpu/nova-core/gsp/boot.rs > +++ b/drivers/gpu/nova-core/gsp/boot.rs > @@ -151,6 +151,33 @@ pub(crate) fn boot( > > Self::run_fwsec_frts(dev, gsp_falcon, bar, &bios, &fb_layout)?; > > + if vgpu_support { > + let scrubber = BooterFirmware::new( > + dev, > + BooterKind::Scrubber, > + chipset, > + FIRMWARE_VERSION, > + sec2_falcon, > + bar, > + )?; > + > + sec2_falcon.reset(bar)?; > + sec2_falcon.dma_load(bar, &scrubber)?; > + > + let (mbox0, mbox1) = sec2_falcon.boot(bar, None, None)?; > + > + dev_dbg!( > + pdev.as_ref(), I think you can use `dev` here? Dirk > + "SEC2 MBOX0: {:#x}, MBOX1{:#x}\n", > + mbox0, > + mbox1 > + ); > + > + if !regs::NV_PGC6_BSI_SECURE_SCRATCH_15::read(bar).scrubber_completed() { > + return Err(ETIMEDOUT); > + } > + } > + > let booter_loader = BooterFirmware::new( > dev, > BooterKind::Loader, > diff --git a/drivers/gpu/nova-core/regs.rs b/drivers/gpu/nova-core/regs.rs > index 82cc6c0790e5..9f3a52ca014f 100644 > --- a/drivers/gpu/nova-core/regs.rs > +++ b/drivers/gpu/nova-core/regs.rs > @@ -173,6 +173,17 @@ pub(crate) fn higher_bound(self) -> u64 { > 26:26 boot_stage_3_handoff as bool; > }); > > +register!(NV_PGC6_BSI_SECURE_SCRATCH_15 @ 0x001180fc { > + 31:29 scrubber_handoff as u8; > +}); > + > +impl NV_PGC6_BSI_SECURE_SCRATCH_15 { > + /// Returns `true` if scrubber is completed. > + pub(crate) fn scrubber_completed(self) -> bool { > + self.scrubber_handoff() >= 0x3 > + } > +} > + > // Privilege level mask register. It dictates whether the host CPU has privilege to access the > // `PGC6_AON_SECURE_SCRATCH_GROUP_05` register (which it needs to read GFW_BOOT). > register!(NV_PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK @ 0x00118128,