From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B215437BE71 for ; Tue, 28 Apr 2026 17:24:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777397101; cv=none; b=NYF/9X1SsVSceb+M1hOTWiGZsCMYzRWrPD4etSrJrdSQZRlc3C7+ya3enwUqr6md7qoKet7H0GaKGvSSksV3Y7P1A+2m/Zl/FA3wskYRjme6jm3+Br7MrMOvGGSBIge5XwaOGu4FhO5nRDdXfR8Vt2OqDvecHkW8EHkp95sK1e4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777397101; c=relaxed/simple; bh=exLRwTo+54W3A66dnpUJq+YzGvJxDAtlrClsBtQ/wMY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=TXXvkw2KNpiz0N/kVIFT+0uCsjw2DTGlAGlKDnUJJw6lJIqDLpa8RXzBk7b0EFpB30Rpf/lfvGl1ObCN6VbM0t3igamfTH+ObAm+NBxBhB3LEDWSMYawkpY5BGb6wPR0Sah0ZpzNBF49vQpkfp3SoDuy3PPd12z7at5zfAREFBw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EiNFXs4m; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EiNFXs4m" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2b2591757fbso23565ad.0 for ; Tue, 28 Apr 2026 10:24:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777397099; x=1778001899; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=3rrqnMSkJK7r/cDlRpksXyGq7DpXCCWEbuMA7lgNfeg=; b=EiNFXs4mMv/ICvlxdPMxq9IUVnonABUioUTpnGgIetKCyFPF3hleSEplN61VSTNS0s 2H0SR7pmRH49cr/h7xx+gRaKgJubCX40jB+YNPwJ1P7Jpf1bW6s9ZENotAT/bAk4B00h RihmKffoBzBmQIzYzfbKA6IJjrBBFxA0RpIEz3LFNpSCMcyMX8Sox4T8vkwr6QfRtM7g dl3fshctDcZUEXWiQ6hGyoDKR9B6bsFj2BXTThRFdFT60rISMZut3yZ1rAgx9mtSrxiv XWZvl7FZAsMAA+YQlfYINfMV+UEuKd4urizbtABTD10yAldMY5SUwVS5GShhsdxLKXmB GqUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777397099; x=1778001899; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3rrqnMSkJK7r/cDlRpksXyGq7DpXCCWEbuMA7lgNfeg=; b=W83TneVfPHcA16722rARznbofZeijkZ8sDmO0TMGXaZ7MMOMhTSI0WsqaZ1+40xLNj FGiH72lw1wMrRqMWwdrXI+NJWc8iu2QhALXl/y1WcZzMEYNc29a6NfMFU9W3EmIYRA4r EGV0cdljQgmouc+VqKGiIBpAq3Hs2D1pRvCkL96rAapreQDrdGPGQ2OEmcTAJqg4TMpt L3Qhcx829pS61wCqeKGhm85s0fMzk9JtAbKF99aKxqDbsnucI08qG+fHer0bLOya48gj HgjUD1NFQInHdf/Z90z62GwTNv1waT76xI3ffcggyFQ0uJyaOAIxaU1ZygoXku5ZW6hi 0Ymw== X-Forwarded-Encrypted: i=1; AFNElJ9QFDYsob76h12naE/CnCACcjjv8QbpF1NrsA2311q6ZwCDkdofhjXfJd46vSjOmAfYUCzDC8Tx/hsLHb8=@vger.kernel.org X-Gm-Message-State: AOJu0YyVKBe5ReMegJxBSscXT4pUrlf4xIXB5iIMAjI339b/uQeTk3Z+ 6nN3WKQ2a+MhvPj5Ct15wnpmjR2YahkiIiPAZ7TrvVRVVGRmNh9Ut9+twnYFYlfp7Q== X-Gm-Gg: AeBDievamGc0q9Fn0hbt9m4deKQ2xTYz/uGZWqcdV+qn0FAaEim5/DA2gS4wenFtmSC THJffyyl5YkHqjYj1jweee6FXQ8GEwup/qF/OndNwoKMXaVpnJPUeWBD4uddV0Zh4b3l82DAVzX tpnHjpsLS+JjdhR8ach6fVmOjuQxUYdeNAGBONr88Ol6P0oFsFT+2HqvPyYwIutzq91gLUfnlbs BkR7ueZW7gbbUuaOBAOyO3IcJmMg8yThKKjZVDr3L5qGGGsOXy9Wr8G58fzF/u/Jnv5ddaTdQeB CtI7S9HvoRTCx9qUHCSdYg8/R/d3vZRMBEhWCIX8xkf1eBZuwFrYi28FnMKaNDVQkFBoUPiT3ok WJvZ7HekyFIMDi+WX/5CL+666XsLkc7dAOtnYCYqltMCd/IdzmXugI9RYeJlkq0JkmJjBFZR8yx xvGWJITEicUgyZTWJS1N5MLVnMNppvBQpzEdkRScKwYXgXtuszbOD2pZGavx2C8FeUYdUJic0bB vAGe3EHgQQ= X-Received: by 2002:a17:903:380e:b0:2ae:80a3:98a9 with SMTP id d9443c01a7336-2b987f6d05dmr24355ad.11.1777397097903; Tue, 28 Apr 2026 10:24:57 -0700 (PDT) Received: from google.com (195.236.83.34.bc.googleusercontent.com. [34.83.236.195]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2b97aca8369sm31190845ad.75.2026.04.28.10.24.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Apr 2026 10:24:57 -0700 (PDT) Date: Tue, 28 Apr 2026 17:24:53 +0000 From: Samiullah Khawaja To: David Matlack Cc: iommu@lists.linux.dev, kexec@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Adithya Jayachandran , Alexander Graf , Alex Williamson , Bjorn Helgaas , Chris Li , David Rientjes , Jacob Pan , Jason Gunthorpe , Joerg Roedel , Jonathan Corbet , Josh Hilke , Leon Romanovsky , Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Pranjal Shrivastava , Pratyush Yadav , Robin Murphy , Saeed Mahameed , Shuah Khan , Will Deacon , William Tu , Yi Liu Subject: Re: [PATCH v4 02/11] PCI: liveupdate: Track outgoing preserved PCI devices Message-ID: References: <20260423212316.3431746-1-dmatlack@google.com> <20260423212316.3431746-3-dmatlack@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <20260423212316.3431746-3-dmatlack@google.com> On Thu, Apr 23, 2026 at 09:23:06PM +0000, David Matlack wrote: >Add APIs to allow drivers to notify the PCI core of which devices are >being preserved across a Live Update for the next kernel, i.e. >"outgoing" devices. > >Drivers must notify the PCI core when devices are preserved so that the >PCI core can update its FLB data (struct pci_ser) and track the list of >outgoing devices. pci_liveupdate_preserve() notifies the PCI core that a >device must be preserved across Live Update. pci_liveupdate_unpreserve() >reverses this (cancels the preservation of the device). > >This tracking ensures the PCI core is fully aware of which devices may >need special handling during shutdown and kexec, and so that it can be >handed off to the next kernel. > >Signed-off-by: David Matlack >--- > drivers/pci/liveupdate.c | 101 ++++++++++++++++++++++++++++++++++++ > include/linux/kho/abi/pci.h | 7 +-- > include/linux/pci.h | 26 ++++++++++ > 3 files changed, 131 insertions(+), 3 deletions(-) > >diff --git a/drivers/pci/liveupdate.c b/drivers/pci/liveupdate.c >index d4fa61625d56..2dd8daa2f17c 100644 >--- a/drivers/pci/liveupdate.c >+++ b/drivers/pci/liveupdate.c >@@ -43,6 +43,26 @@ > * > * * ``pci_liveupdate_register_flb(driver_file_handler)`` > * * ``pci_liveupdate_unregister_flb(driver_file_handler)`` >+ * >+ * Device Tracking >+ * =============== >+ * >+ * Drivers must notify the PCI core when specific devices are preserved or >+ * unpreserved with the following APIs: >+ * >+ * * ``pci_liveupdate_preserve(pci_dev)`` >+ * * ``pci_liveupdate_unpreserve(pci_dev)`` >+ * >+ * This allows the PCI core to keep it's FLB data (struct pci_ser) up to date >+ * with the list of **outgoing** preserved devices for the next kernel. >+ * >+ * Restrictions >+ * ============ >+ * >+ * The PCI core enforces the following restrictions on which devices can be >+ * preserved. These may be relaxed in the future: >+ * >+ * * The device cannot be a Virtual Function (VF). > */ > > #define pr_fmt(fmt) "PCI: liveupdate: " fmt >@@ -57,6 +77,8 @@ > #include > #include > >+static DEFINE_MUTEX(pci_flb_outgoing_lock); >+ > static int pci_flb_preserve(struct liveupdate_flb_op_args *args) > { > struct pci_dev *dev = NULL; >@@ -124,6 +146,85 @@ static struct liveupdate_flb pci_liveupdate_flb = { > .compatible = PCI_LUO_FLB_COMPATIBLE, > }; > >+int pci_liveupdate_preserve(struct pci_dev *dev) >+{ >+ struct pci_ser *ser; >+ int i, ret; >+ >+ guard(mutex)(&pci_flb_outgoing_lock); >+ >+ ret = liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser); >+ if (ret) >+ return ret; >+ >+ if (!ser) >+ return -ENOENT; >+ >+ if (dev->is_virtfn) >+ return -EINVAL; >+ >+ if (dev->liveupdate_outgoing) >+ return -EBUSY; >+ >+ if (ser->nr_devices == ser->max_nr_devices) >+ return -ENOSPC; >+ >+ for (i = 0; i < ser->max_nr_devices; i++) { >+ /* >+ * Start searching at index ser->nr_devices. This should result >+ * in a constant time search under expected conditions (devices >+ * are not getting unpreserved). >+ */ >+ int index = (ser->nr_devices + i) % ser->max_nr_devices; >+ struct pci_dev_ser *dev_ser = &ser->devices[index]; nit: Maybe we can move this logic in a separate function as down the road when we expand this to add VFs and Hotpluggable devices, this might change significantly? It's good if it is self-contained. >+ >+ if (dev_ser->refcount) >+ continue; >+ >+ pci_info(dev, "Device will be preserved across next Live Update\n"); >+ ser->nr_devices++; >+ >+ dev_ser->domain = pci_domain_nr(dev->bus); >+ dev_ser->bdf = pci_dev_id(dev); >+ dev_ser->refcount = 1; >+ >+ dev->liveupdate_outgoing = dev_ser; >+ return 0; >+ } >+ >+ return -ENOSPC; >+} >+EXPORT_SYMBOL_GPL(pci_liveupdate_preserve); >+ >+void pci_liveupdate_unpreserve(struct pci_dev *dev) >+{ >+ struct pci_dev_ser *dev_ser; >+ struct pci_ser *ser = NULL; >+ int ret; >+ >+ guard(mutex)(&pci_flb_outgoing_lock); >+ >+ ret = liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser); >+ >+ if (ret || !ser) { >+ pci_warn(dev, "Cannot unpreserve device without outgoing Live Update state\n"); >+ return; >+ >+ } >+ >+ dev_ser = dev->liveupdate_outgoing; >+ if (!dev_ser) { >+ pci_warn(dev, "Cannot unpreserve device that is not preserved\n"); >+ return; >+ } >+ >+ pci_info(dev, "Device will no longer be preserved across next Live Update\n"); >+ ser->nr_devices--; >+ memset(dev_ser, 0, sizeof(*dev_ser)); >+ dev->liveupdate_outgoing = NULL; >+} >+EXPORT_SYMBOL_GPL(pci_liveupdate_unpreserve); >+ > int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh) > { > pr_debug("Registering file handler \"%s\"\n", fh->compatible); >diff --git a/include/linux/kho/abi/pci.h b/include/linux/kho/abi/pci.h >index 5c0e92588c00..5b4c8d9e462c 100644 >--- a/include/linux/kho/abi/pci.h >+++ b/include/linux/kho/abi/pci.h >@@ -23,19 +23,20 @@ > * incrementing the version number in the PCI_LUO_FLB_COMPATIBLE string. > */ > >-#define PCI_LUO_FLB_COMPATIBLE "pci-v1" >+#define PCI_LUO_FLB_COMPATIBLE "pci-v2" > > /** > * struct pci_dev_ser - Serialized state about a single PCI device. > * > * @domain: The device's PCI domain number (segment). > * @bdf: The device's PCI bus, device, and function number. >- * @reserved: Reserved (to naturally align struct pci_dev_ser). >+ * @refcount: Reference count used by the PCI core to keep track of whether it >+ * is done using a device's struct pci_dev_ser. > */ > struct pci_dev_ser { > u32 domain; > u16 bdf; >- u16 reserved; >+ u16 refcount; > } __packed; > > /** >diff --git a/include/linux/pci.h b/include/linux/pci.h >index d70080babd52..eb94cbd8ab9d 100644 >--- a/include/linux/pci.h >+++ b/include/linux/pci.h >@@ -41,6 +41,7 @@ > #include > #include > #include >+#include > > #include > >@@ -594,6 +595,9 @@ struct pci_dev { > u8 tph_mode; /* TPH mode */ > u8 tph_req_type; /* TPH requester type */ > #endif >+#ifdef CONFIG_PCI_LIVEUPDATE >+ struct pci_dev_ser *liveupdate_outgoing; /* State preserved for next kernel */ >+#endif > }; > > static inline struct pci_dev *pci_physfn(struct pci_dev *dev) >@@ -2880,6 +2884,14 @@ void pci_uevent_ers(struct pci_dev *pdev, enum pci_ers_result err_type); > #ifdef CONFIG_PCI_LIVEUPDATE > int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh); > void pci_liveupdate_unregister_flb(struct liveupdate_file_handler *fh); >+ >+int pci_liveupdate_preserve(struct pci_dev *dev); >+void pci_liveupdate_unpreserve(struct pci_dev *dev); >+ >+static inline struct pci_dev_ser *pci_liveupdate_outgoing(struct pci_dev *dev) >+{ >+ return dev->liveupdate_outgoing; >+} Is this expected to be called under the outgoing lock? > #else > static inline int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh) > { >@@ -2889,6 +2901,20 @@ static inline int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh > static inline void pci_liveupdate_unregister_flb(struct liveupdate_file_handler *fh) > { > } >+ >+static inline int pci_liveupdate_preserve(struct pci_dev *dev) >+{ >+ return -EOPNOTSUPP; >+} >+ >+static inline void pci_liveupdate_unpreserve(struct pci_dev *dev) >+{ >+} >+ >+static inline struct pci_dev_ser *pci_liveupdate_outgoing(struct pci_dev *dev) >+{ >+ return NULL; >+} > #endif > > #endif /* LINUX_PCI_H */ >-- >2.54.0.rc2.544.gc7ae2d5bb8-goog >