From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4BFA1C43458 for ; Fri, 10 Jul 2026 21:26:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=nJNVxjvVmYROzvILHkxiv/3zu6J6S8o6lQRGVaWTNbg=; b=uIOvLKCXyJ+A4oY7PjZ72oyftv a8D+YHpBH6TdqKUPbXf6tMaUPAHUoCb7yNHyP6iZ2G4ZXbMzVagNmSwfub9mkid5ONYnuVCMwg+S9 20yoLhpTFKRfAAGtAveGboov/6RX1asAv3/7JrDYa1oXCwxWQvLxW5U6F1xWgLKv6LZ9XttAb+anC eGRL/ERvFJ9Eiua6L6mbkQDwW7NajfaaTuqD+EYPFlv8mTjXWmeNswIp4dU3jyhaLY/51BtN+alBb e3RjaAy5FwsWXTNltB1cyqv7vU2pjHY+3Q6kXC8PtCBv0eaInd0q0LAJB1yz+edj6T4IEeNQ5sUjd OQOSdCvg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wiIjr-00000005pMr-2mAR; Fri, 10 Jul 2026 21:26:27 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wiIjp-00000005pLY-2519 for kexec@lists.infradead.org; Fri, 10 Jul 2026 21:26:26 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-8486ffba174so2088156b3a.1 for ; Fri, 10 Jul 2026 14:26:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1783718784; x=1784323584; darn=lists.infradead.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=nJNVxjvVmYROzvILHkxiv/3zu6J6S8o6lQRGVaWTNbg=; b=jDj9P8NCkIVfr7fKLxVJyGuKUdToUE0FV4SACyp3rPhC3Mwge0sDripVN8U5TZBEUL dT8I6hTQZ01f5TzrKwIRcwwVSW0RKklf0htC3WbKXb63igzpt3hKtYO9KknWEZrZkOix 9uMN0VOAb5bwwo0VLuAz522q1mkpa4cu2vvKCkMYk+uuh99lUyEvb/a/DO03RciSYN1k VHNfxrnrQewkA9LpWf6v04ajG1+WuLUQQ7J93vvwr132KiZVvKSndJFq99wdfVI69mko LOltbPj6lBmfH2GAesYdckkZcXxxi+i4sbvAfxQ7VCMfOiRHuuxmYHniHr/i7hjXA7GW /hpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783718784; x=1784323584; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=nJNVxjvVmYROzvILHkxiv/3zu6J6S8o6lQRGVaWTNbg=; b=BDMMx81AcLc9eL6tfKrZfGaastmGSDHHh9AUx7DZG2xxtukxGAZrspzDnK58MEiy/M 6HvfiWHnFQ6APJFkXkv/3Gx07EPGcG0CuNMWRx8cR9iU6yGxKe6EifaUCq3PMkQzdAHl 82E0ImMGgCuXseceaju9SqjPe0oVaPIzrCvLarEAYYtmrhFiDnFTSkeaA1GLfIO5J0qX Zu2vmiEi2n9KOm6MUmskc2iJrQMZtuxKSRYl/JvaEsvsSJ1d87g34oSqvSvpFQA0ObYE JnWyDn4oOSIM7QWyGjYozKDg53EHVNIsdwDmGU1S7C4IC2cDcKC3eEc3qThyEaDePrYk 3PvQ== X-Gm-Message-State: AOJu0YwZykmpYGFdrEtVGCtBioM1MozauU6tiAD+ww5op6tdyWton2wm 7lT6eZG3tVKc6LlnKcDuwJPEttmpd8EMULbwhN1eYRYm54FoSmma/s12ZlS6/0h0M8E708RiIaf eGGlvWrnn4nX//PV2ndG0UDKFJQUS7ED8ebQfwbIRJ5f1BuJCNIRf5BoZOjv6FOJr/wQXecyz1Q vOxelV4n2bNTb7/W9AV/pe5fCgnLkJTq82xAgc6dtYN0Fo4+W3Cy8= X-Received: from pfbln8.prod.google.com ([2002:a05:6a00:3cc8:b0:848:48a0:41e6]) (user=dmatlack job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:179b:b0:848:30fe:da34 with SMTP id d2e1a72fcca58-8488974b074mr589334b3a.45.1783718783985; Fri, 10 Jul 2026 14:26:23 -0700 (PDT) Date: Fri, 10 Jul 2026 21:26:04 +0000 In-Reply-To: <20260710212616.1351130-1-dmatlack@google.com> Mime-Version: 1.0 References: <20260710212616.1351130-1-dmatlack@google.com> X-Mailer: git-send-email 2.55.0.795.g602f6c329a-goog Message-ID: <20260710212616.1351130-2-dmatlack@google.com> Subject: [PATCH v7 01/12] PCI: liveupdate: Set up FLB handler for the PCI core From: David Matlack To: kexec@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org Cc: Adithya Jayachandran , Alexander Graf , Alex Williamson , Bjorn Helgaas , Chris Li , David Matlack , David Rientjes , Jacob Pan , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Leon Romanovsky , Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Pranjal Shrivastava , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , William Tu , Yi Liu Content-Type: text/plain; charset="UTF-8" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260710_142625_550676_8AFD2198 X-CRM114-Status: GOOD ( 34.74 ) X-BeenThere: kexec@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kexec" Errors-To: kexec-bounces+kexec=archiver.kernel.org@lists.infradead.org Set up a File-Lifecycle-Bound (FLB) handler for the PCI core to enable it to participate in the preservation of PCI devices across Live Update. Essentially, this commit enables the PCI core to allocate a struct (struct pci_ser) and preserve it across a Live Update whenever at least one device is preserved. Preserving PCI devices across Live Update is built on top of the Live Update Orchestrator's (LUO) support for file preservation. Drivers are expected to expose a file to userspace to represent a single PCI device and support preservation of that file. This is intended primarily to support preservation of PCI devices bound to VFIO drivers. This commit enables drivers to register their liveupdate_file_handler with the PCI core so that the PCI core can do its own tracking and enforcement of which devices are preserved. pci_liveupdate_register_flb(driver_file_handler); pci_liveupdate_unregister_flb(driver_file_handler); When the first file (with a handler registered with the PCI core) is preserved, the PCI core will be notified to allocate its tracking struct (pci_ser). When the last file is unpreserved (i.e. preservation cancelled) the PCI core will be notified to free struct pci_ser. This struct is preserved across a Live Update using KHO and can be fetched by the PCI core during early boot (e.g. during device enumeration) so that it knows which devices were preserved. Note: This commit only allocates struct pci_ser and preserves it across Live Update. A subsequent commit will add an API for drivers to tell the PCI core exactly which devices are being preserved. Note: There is no reason to check for kho_is_enabled() since it can be assumed to return true. If KHO was not enabled then Live Update would not be enabled and these routines would never run. Reviewed-by: Pranjal Shrivastava Signed-off-by: David Matlack --- Documentation/core-api/liveupdate.rst | 4 + MAINTAINERS | 13 +++ drivers/pci/Kconfig | 15 +++ drivers/pci/Makefile | 1 + drivers/pci/liveupdate.c | 155 ++++++++++++++++++++++++++ include/linux/kho/abi/pci.h | 58 ++++++++++ include/linux/pci.h | 1 + include/linux/pci_liveupdate.h | 30 +++++ 8 files changed, 277 insertions(+) create mode 100644 drivers/pci/liveupdate.c create mode 100644 include/linux/kho/abi/pci.h create mode 100644 include/linux/pci_liveupdate.h diff --git a/Documentation/core-api/liveupdate.rst b/Documentation/core-api/liveupdate.rst index 5a292d0f3706..b3c689e633c1 100644 --- a/Documentation/core-api/liveupdate.rst +++ b/Documentation/core-api/liveupdate.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: GPL-2.0 +.. _luo: + ======================== Live Update Orchestrator ======================== @@ -18,6 +20,8 @@ LUO Preserving File Descriptors .. kernel-doc:: kernel/liveupdate/luo_file.c :doc: LUO File Descriptors +.. _flb: + LUO File Lifecycle Bound Global Data ==================================== .. kernel-doc:: kernel/liveupdate/luo_flb.c diff --git a/MAINTAINERS b/MAINTAINERS index a3ed337e827d..9cc7b9291ace 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -20824,6 +20824,19 @@ L: linux-pci@vger.kernel.org S: Supported F: Documentation/PCI/pci-error-recovery.rst +PCI LIVE UPDATE +M: David Matlack +R: Pasha Tatashin +R: Mike Rapoport +R: Pratyush Yadav +L: kexec@lists.infradead.org +L: linux-pci@vger.kernel.org +S: Maintained +T: git git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux.git +F: drivers/pci/liveupdate.c +F: include/linux/kho/abi/pci.h +F: include/linux/pci_liveupdate.h + PCI MSI DRIVER FOR ALTERA MSI IP L: linux-pci@vger.kernel.org S: Orphan diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 0c7408509ba2..3781e2b5f095 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -271,6 +271,21 @@ config VGA_ARB_MAX_GPUS Reserves space in the kernel to maintain resource locking for multiple GPUS. The overhead for each GPU is very small. +config PCI_LIVEUPDATE + bool "PCI Live Update Support" + depends on PCI && LIVEUPDATE + help + Enable PCI core support for preserving PCI devices across Live + Update. This, in combination with support in a device's driver, + enables PCI devices to run and perform memory transactions + uninterrupted during a kexec for Live Update. + + This option should only be enabled by users who plan to use Live + Update for kernel upgrades and require preserving PCI devices during + those upgrades. + + If unsure, say N. + source "drivers/pci/hotplug/Kconfig" source "drivers/pci/controller/Kconfig" source "drivers/pci/endpoint/Kconfig" diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile index 41ebc3b9a518..e8d003cb6757 100644 --- a/drivers/pci/Makefile +++ b/drivers/pci/Makefile @@ -16,6 +16,7 @@ obj-$(CONFIG_PROC_FS) += proc.o obj-$(CONFIG_SYSFS) += pci-sysfs.o slot.o obj-$(CONFIG_ACPI) += pci-acpi.o obj-$(CONFIG_GENERIC_PCI_IOMAP) += iomap.o +obj-$(CONFIG_PCI_LIVEUPDATE) += liveupdate.o endif obj-$(CONFIG_OF) += of.o diff --git a/drivers/pci/liveupdate.c b/drivers/pci/liveupdate.c new file mode 100644 index 000000000000..899758883dd5 --- /dev/null +++ b/drivers/pci/liveupdate.c @@ -0,0 +1,155 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright (c) 2026, Google LLC. + * David Matlack + */ + +/** + * DOC: PCI Live Update + * + * The PCI subsystem participates in the Live Update process to enable drivers + * to preserve their PCI devices across kexec. + * + * :ref:`FLB ` Data + * ===================== + * + * PCI device preservation across Live Update is built on top of the + * :ref:`LUO ` support for file preservation across kexec. Drivers are + * expected to expose a file to represent a single PCI device and support + * preservation of that file with ``ioctl(LIVEUPDATE_SESSION_PRESERVE_FD)``. + * This allows userspace to control the preservation of devices and ensure + * proper lifecycle management while a device is preserved. The first intended + * use-case is preserving vfio-pci device files. + * + * The PCI core maintains its own state about what devices are being preserved + * across Live Update using FLB data in LUO. Essentially, this allows the PCI + * core to allocate struct pci_ser when the first device (file) is preserved + * and free it when the last device (file) is unpreserved. After kexec, the + * PCI core can fetch the struct pci_ser (which was constructed by the previous + * kernel) from LUO at any time (e.g. during enumeration) so that it knows + * which devices were preserved. + * + * To enable the PCI core to be notified whenever a file representing a device + * is preserved, drivers must register their struct liveupdate_file_handler with + * the PCI core by using the following APIs: + * + * * ``pci_liveupdate_register_flb(driver_file_handler)`` + * * ``pci_liveupdate_unregister_flb(driver_file_handler)`` + */ + +#define pr_fmt(fmt) "PCI: " KBUILD_BASENAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/** + * struct pci_flb_outgoing - Outgoing PCI FLB object + * @ser: Pointer to the preserved struct pci_ser. + * @block_set: The KHO block set holding the outgoing devices. + * + * This structure holds the runtime state for the outgoing PCI Live Update + * state. It wraps the serialized pci_ser and the block_set used to manage + * the serialized entries. + */ +struct pci_flb_outgoing { + struct pci_ser *ser; + struct kho_block_set block_set; +}; + +static int pci_flb_preserve(struct liveupdate_flb_op_args *args) +{ + struct pci_flb_outgoing *outgoing; + struct pci_ser *ser; + + outgoing = kzalloc_obj(*outgoing); + if (!outgoing) + return -ENOMEM; + + ser = kho_alloc_preserve(sizeof(*ser)); + if (IS_ERR(ser)) { + kfree(outgoing); + return PTR_ERR(ser); + } + + ser->nr_devices = 0; + ser->devices = 0; + + outgoing->ser = ser; + kho_block_set_init(&outgoing->block_set, sizeof(struct pci_dev_ser)); + + args->obj = outgoing; + args->data = virt_to_phys(ser); + return 0; +} + +static void pci_flb_unpreserve(struct liveupdate_flb_op_args *args) +{ + struct pci_flb_outgoing *outgoing = args->obj; + + WARN_ON(outgoing->ser->nr_devices); + kho_block_set_destroy(&outgoing->block_set); + kho_unpreserve_free(outgoing->ser); + kfree(outgoing); + pr_debug("Unpreserved struct pci_ser\n"); +} + +static int pci_flb_retrieve(struct liveupdate_flb_op_args *args) +{ + args->obj = phys_to_virt(args->data); + return 0; +} + +static void pci_flb_finish(struct liveupdate_flb_op_args *args) +{ + kho_restore_free(args->obj); +} + +static struct liveupdate_flb_ops pci_liveupdate_flb_ops = { + .preserve = pci_flb_preserve, + .unpreserve = pci_flb_unpreserve, + .retrieve = pci_flb_retrieve, + .finish = pci_flb_finish, + .owner = THIS_MODULE, +}; + +static struct liveupdate_flb pci_liveupdate_flb = { + .ops = &pci_liveupdate_flb_ops, + .compatible = PCI_LUO_FLB_COMPATIBLE, +}; + +/** + * pci_liveupdate_register_flb() - Register a file handler with the PCI core + * @fh: The file handler to register. + * + * Drivers should call pci_liveupdate_register_flb() to register their + * struct liveupdate_file_handler with the PCI core. This enables the PCI core + * to allocate its outgoing struct pci_ser whenever the first device is + * preserved, and free it when the last device is unpreserved. + * + * Return: 0 on success, <0 on failure. + */ +int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh) +{ + pr_debug("Registering file handler \"%s\"\n", fh->compatible); + return liveupdate_register_flb(fh, &pci_liveupdate_flb); +} +EXPORT_SYMBOL_GPL(pci_liveupdate_register_flb); + +/** + * pci_liveupdate_unregister_flb() - Unregister a file handler with the PCI core + * @fh: The file handler to unregister. + */ +void pci_liveupdate_unregister_flb(struct liveupdate_file_handler *fh) +{ + pr_debug("Unregistering file handler \"%s\"\n", fh->compatible); + liveupdate_unregister_flb(fh, &pci_liveupdate_flb); +} +EXPORT_SYMBOL_GPL(pci_liveupdate_unregister_flb); diff --git a/include/linux/kho/abi/pci.h b/include/linux/kho/abi/pci.h new file mode 100644 index 000000000000..de549016807a --- /dev/null +++ b/include/linux/kho/abi/pci.h @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Copyright (c) 2026, Google LLC. + * David Matlack + */ + +#ifndef _LINUX_KHO_ABI_PCI_H +#define _LINUX_KHO_ABI_PCI_H + +#include +#include +#include + +/** + * DOC: PCI File-Lifecycle Bound (FLB) Live Update ABI + * + * This header defines the ABI for preserving core PCI state across kexec using + * Live Update File-Lifecycle Bound (FLB) data. + * + * This interface is a contract. Any modification to any of the serialization + * structs defined here constitutes a breaking change. Such changes require + * incrementing the version number in the PCI_LUO_FLB_COMPATIBLE string. + */ + +#define PCI_LUO_FLB_COMPATIBLE "pci-v1" + +/** + * struct pci_dev_ser - Serialized state about a single PCI device. + * + * @domain: The device's PCI domain number (segment). + * @bdf: The device's PCI bus, device, and function number. + * @refcount: Reference count used by the PCI core to keep track of whether it + * is done using a device's struct pci_dev_ser. The value of the + * refcount is equal to 1 when the struct pci_dev_ser is in use, and + * 0 otherwise. + */ +struct pci_dev_ser { + u32 domain; + u16 bdf; + u16 refcount; +} __packed; + +/** + * struct pci_ser - PCI Subsystem Live Update State + * + * This struct tracks state about all devices that are being preserved across + * a Live Update for the next kernel. + * + * @nr_devices: The number of devices that were preserved. + * @devices: Physical address of the first KHO block containing pci_dev_ser. + */ +struct pci_ser { + u32 nr_devices; + u64 devices; +} __packed; + +#endif /* _LINUX_KHO_ABI_PCI_H */ diff --git a/include/linux/pci.h b/include/linux/pci.h index ebb5b9d76360..da58aa101e4c 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -43,6 +43,7 @@ #include #include +#include #define PCI_STATUS_ERROR_BITS (PCI_STATUS_DETECTED_PARITY | \ PCI_STATUS_SIG_SYSTEM_ERROR | \ diff --git a/include/linux/pci_liveupdate.h b/include/linux/pci_liveupdate.h new file mode 100644 index 000000000000..8ec98beefcb4 --- /dev/null +++ b/include/linux/pci_liveupdate.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * PCI Live Update support (Public/Driver API) + * + * Copyright (c) 2026, Google LLC. + * David Matlack + */ +#ifndef LINUX_PCI_LIVEUPDATE_H +#define LINUX_PCI_LIVEUPDATE_H + +#include +#include + +struct pci_dev; + +#ifdef CONFIG_PCI_LIVEUPDATE +int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh); +void pci_liveupdate_unregister_flb(struct liveupdate_file_handler *fh); +#else +static inline int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh) +{ + return -EOPNOTSUPP; +} + +static inline void pci_liveupdate_unregister_flb(struct liveupdate_file_handler *fh) +{ +} +#endif + +#endif /* LINUX_PCI_LIVEUPDATE_H */ -- 2.55.0.795.g602f6c329a-goog