From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9CAB2C98304 for ; Wed, 23 Sep 2026 23:28:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A85346B0092; Wed, 23 Sep 2026 19:28:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A367C6B0093; Wed, 23 Sep 2026 19:28:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 924EF6B0095; Wed, 23 Sep 2026 19:28:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6D06E6B0092 for ; Wed, 23 Sep 2026 19:28:12 -0400 (EDT) Received: from smtpin29.hostedemail.com (lb01a-stub [10.200.18.249]) by unirelay04.hostedemail.com (Postfix) with ESMTP id DA8C61A0115 for ; Wed, 23 Sep 2026 23:28:11 +0000 (UTC) X-FDA: 85246617582.29.F479D07 Received: from mta1.migadu.com (out-85.mta1.migadu.com [95.215.58.85]) by imf04.hostedemail.com (Postfix) with ESMTP id F04A440002 for ; Wed, 23 Sep 2026 23:28:08 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="p/tjJQbH"; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf04.hostedemail.com: domain of yanjun.zhu@linux.dev designates 95.215.58.85 as permitted sender) smtp.mailfrom=yanjun.zhu@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1790206089; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OIrOn8xctsdQQDNeQdRku02Zq+U+2COm+blhhp9fc/s=; b=GwA4T2FD6BjLAkwVLIEXDuo8Uw0ycrvOzYnp9kjCUjUZm5gq6Que1ARcA33YWM7SOTg40Z 9Bxn2gdUXN1mHTlTC4cIvtA0PZjwlNWJ1cuJH873URYosMJ1yvUHWfUXRh85QI66jcxw+C g1l9SkZ8Pc0MPZPdL+fp6ADXszxjCvw= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="p/tjJQbH"; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf04.hostedemail.com: domain of yanjun.zhu@linux.dev designates 95.215.58.85 as permitted sender) smtp.mailfrom=yanjun.zhu@linux.dev ARC-Seal: i=1; a=rsa-sha256; d=hostedemail.com; s=arc-20220608; cv=none; t=1790206089; b=2/AuXyIrQqRbAdpNqT7sFJXjom731bcAQUt57XafqC0NPFroQaExUpKzvohImg5zdx0jon fC8puJoRqmDd2mz+5JLx4uI3ixFe5IE2F2uMUGE4fdf9okbsJdQFzlgXq21m5IUva5ER5h ICG50HF/vpnrYTvaBI3RFLg7z15m7TU= X-Envelope-To: linux-mm@kvack.org DKIM-Signature: a=rsa-sha256; bh=MvumBShhqXhAx5A4XnSnlBOd7T1e+nrArHC0D0D2ILE=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1790206084; v=1; x=1790810884; b=p/tjJQbH1r+Mu4nL+WUVU8yNeG/vzL3ZUqHLb4IqRFIVu4VN5T+AEiNOWvIYKZ0gLmhPefJH awaqW6uhHo3Lhy2dHoH8L6XXjQXSXvVhsPlqcvOwc58DeEVCznc3QzGnVpWH9d9scyQL7UYBgSC QGSZyVZ30ekzOg1481IxbJ/A= X-Envelope-To: linux-mm@kvack.org Received: by smtp.migadu.com with ESMTPS id 9e4aaf2a06ca83ad; Wed, 23 Sep 2026 23:28:04 +0000 X-Mizu-Trace-ID: 9e4aaf2a06ca83ad X-Migadu-Flow: FLOW_OUT Content-Type: multipart/alternative; boundary="------------w02yeVEA7OyGJFRzsrh3hdEm" Message-ID: Date: Wed, 23 Sep 2026 16:27:57 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v9 00/13] PCI: liveupdate: PCI core support for Live Update To: David Matlack Cc: kexec@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Adithya Jayachandran , Alexander Graf , Alex Williamson , Bjorn Helgaas , Chris Li , David Rientjes , Jacob Pan , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Leon Romanovsky , Lukas Wunner , Mike Rapoport , Parav Pandit , Pasha Tatashin , Pranjal Shrivastava , Pratyush Yadav , Randy Dunlap , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , William Tu , Yi Liu , "yanjun.zhu@linux.dev" References: <20260918200640.887030-1-dmatlack@google.com> <2e88b92a-d3f2-41f7-bdbe-11fd71690606@linux.dev> From: Zhu Yanjun In-Reply-To: X-Rspam-User: X-Stat-Signature: rmrjymzwqis6h6nsaruhcnazoygwn3yh X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: F04A440002 X-HE-Tag: 1790206088-372386 X-HE-Meta: U2FsdGVkX19bGgQ1LPQdj+pzX8ZX6b17dtCNhhTcHCMxadVCfsVU/utCkzVkIl0yIuOpksL9MPHkyq7qSmeyU9k1Eg8QRLQ3d12PcoCBDI31ZFY3+6bZgV0Y+r4UOD5VcXhFrrYOwtfUXRRfAJz5KovoPz8YARtcxezvNyBIFfxUfuMZzsEuWkeqhAGZ/eo0s2O08PUD5BeLGFG73d6gDmnuHY2ObnZmkVVCQ+qxrX0xX6fwZAU+lOV4yjI0gM4EzlvMYe9TwuwvTripMib+5WJHNR3grLl3+GdImvX/alIcUIeil5qyaf724JyR6g2TzZsiipUeYZ4lwcOWM/oHsPldVCSGR4nHl6Ynyi0PSOdrygAUUsvYIG5Ci8SVtFkG+P4Rwx0aeEWLz6JEFfVDKgRm3i7Cz/2Yvl8Vi6camPQDgy3EFsVyFloXjmA1N5xyduMGchPrfRDLe5DKPdGy9qaJTY1CLnTfzfc9ZMX4uL8V+nU9rA/TwHaMF/S/vZBy6+7d1OIrEA84GvaznBHJP5jerRipFiaK7xObL9X3lOZWaAtnETig4WNERRqMdNKWKYWnX0LDDxr42M2yDjfm7gXO3kJgs8c3GjP7fCRXskggQEmozfCtor/mUQixtNPnDKPID544tXFakzKZ5X+c4GBQpOjvEmJ3oMQyxkXFJThHa8dFnb59Tmi8vbgOc/0kbZ2InR8y17WbYbqI8y3C4XcUr3OXg+kB0BQ7kc4M/FpSY1dcmGIZCCFIiJpdwl3CuIeYoBOei13U1VzGIKqTe4ZjhU0SHLayLJPCvgjs4/l2mWx0Bvg15mTE9/9HZj827Q2PAvLv21Y9qMRxYgDIfqdhgxjbbdGWTyIrksFMdVBoB7zmLyQUNMUcIJoJz0sfrJjC9pnagA6cgNgF6WCgn5sqnt6a1oX3RMxeEnNSXgcq+SWW9ON3x1Kc3tJJE6Xa+5v4FLvDRr0lSk8k0q/ BGuDsy4p YczDyJjdTH8ZWpAit73FpMM+tq9D3zE594/YkgeXtB31Tv1KUr+aCO+Li86FYuCfF7AvTQdx/34LCTeczZkwn9dC+CZuJMm1bdePZBwzDtvbfz374/W3fWBBLvJF7Q+3oj4o8lVUNVNRiYcKchl8ACdcVWuNf9MVWB2iCLzxtbaK7B7O/DZTLTnylZMc2n6Y1Gfw5jZFPPlNUKbs5hZSi6aiMpFG0oD+tVa8who1vr8EG+un2wsSRq9diFWs93CgVqu2Y69r8sJ+zJAFlFl20Qdkw/DGpjJfQZs8wqp79unurcy0FydyJGQJQjNoyM9TAnJQVG7RRZrMumvjlfV38XOQ/FrMA4apN0oPIU0/bMj1cKOqzV2gKirC6NrIxA5QO4JTdd8iAG25IcJ2iJnOvpRGmea5c0HOAiHc40Inneh/ki4gECUxXW+Dm22AsctD0UJ0GZGlZgkCG6HMK3h3deQ3oFWCCoY/HXzoTgXcOP15MNuxtzn6t+w38Ess4WDESbQ5cyYgL7GWfbTGrvNlwYsPGUQ7ad17sQxRTMUbM3AvIf2tyBIsxKA9E0ihMVB3Qc6Nh Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a multi-part message in MIME format. --------------w02yeVEA7OyGJFRzsrh3hdEm Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit 在 2026/9/22 11:53, David Matlack 写道: > On Tue, Sep 22, 2026 at 11:36 AM Zhu Yanjun wrote: >> 在 2026/9/18 13:06, David Matlack 写道: >>> Future Work >>> ----------- >>> >>> Following this series, we expect to make further improvements to the PCI >>> core support for Live Update: >>> >>> - Allow P2P across Live Update by avoiding resizing or moving >>> preserved device BARs and preserving all upstream bridge windows. >>> >>> - Support preserving Virtual Functions by preserving SR-IOV >>> configuration on PFs and enumerating VFs after Live Update. >> Preserving the PCIe topology, bus numbers, ACS, and Bus Mastering across >> kexec is a foundational step for minimizing downtime. >> >> As we look toward complete end-to-end support for DMA preservation >> across Live Update—especially for VFIO device passthrough and dma-buf >> sharing scenarios—IOMMU table/domain preservation becomes crucial to >> prevent IOMMU page faults when devices continue performing DMA during kexec. >> >> I would like to ask about the current status and roadmap regarding IOMMU >> Live Update / KHO (Kexec Handover) support: >> >> Is there an ongoing effort or RFC series for IOMMU handover / page-table >> preservation currently in development or under discussion? >> >> How is the coordination between the PCI core Live Update mechanisms and >> the IOMMU subsystem being envisioned for preserving IOVA mappings (e.g., >> restoring domains or handing over root tables)? >> >> Any pointers to active discussion threads, RFCs, or future plans >> regarding IOMMU participation in Live Update would be greatly appreciated. > The first IOMMU series to support Live update, can be found here: > > https://lore.kernel.org/linux-iommu/20260921004834.2601285-1-skhawaja@google.com/ Thanks. I have a question regarding the restoration sequencing when module dependencies are involved during a Live Update reboot. For the standard hardware/driver stack, the sequence seems to naturally follow the kernel's early initcalls and device probing (e.g., IOMMU early hardware handover -> PCI bus topology/BME preservation -> IOMMU domain attach & DMA ownership claim -> VFIO/iommufd cdev binding). However, if there is a custom kernel module or subsystem (let's call it Module A) that is not part of the standard PCI/IOMMU device probe callback chain, but strictly depends on the fully restored state of PCI, IOMMU, and VFIO/iommufd: 1. What is the recommended or standardized way in the Live Update architecture to guarantee that Module A's restoration happens *after* all its underlying dependencies (PCI / IOMMU / VFIO) have completely finished their restore processes? 2. Is the expectation to rely on LUO (Live Update Orchestrator) phase notification callbacks (e.g., late restore notifiers), Driver Core mechanisms like |-EPROBE_DEFER| / |device_link|, or something else? Any guidance on how cross-subsystem restoration order and async probe dependencies should be handled in the Live Update framework would be greatly appreciated. Thanks, Yanjun Zhu > > The IOMMU series builds on top of the PCI core support (this series) > and the VFIO series linked further up in the cover letter. -- Best Regards, Yanjun.Zhu --------------w02yeVEA7OyGJFRzsrh3hdEm Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit


在 2026/9/22 11:53, David Matlack 写道:
On Tue, Sep 22, 2026 at 11:36 AM Zhu Yanjun <yanjun.zhu@linux.dev> wrote:

在 2026/9/18 13:06, David Matlack 写道:

Future Work
-----------

Following this series, we expect to make further improvements to the PCI
core support for Live Update:

   - Allow P2P across Live Update by avoiding resizing or moving
     preserved device BARs and preserving all upstream bridge windows.

   - Support preserving Virtual Functions by preserving SR-IOV
     configuration on PFs and enumerating VFs after Live Update.

Preserving the PCIe topology, bus numbers, ACS, and Bus Mastering across
kexec is a foundational step for minimizing downtime.

As we look toward complete end-to-end support for DMA preservation
across Live Update—especially for VFIO device passthrough and dma-buf
sharing scenarios—IOMMU table/domain preservation becomes crucial to
prevent IOMMU page faults when devices continue performing DMA during kexec.

I would like to ask about the current status and roadmap regarding IOMMU
Live Update / KHO (Kexec Handover) support:

Is there an ongoing effort or RFC series for IOMMU handover / page-table
preservation currently in development or under discussion?

How is the coordination between the PCI core Live Update mechanisms and
the IOMMU subsystem being envisioned for preserving IOVA mappings (e.g.,
restoring domains or handing over root tables)?

Any pointers to active discussion threads, RFCs, or future plans
regarding IOMMU participation in Live Update would be greatly appreciated.

The first IOMMU series to support Live update, can be found here:

  https://lore.kernel.org/linux-iommu/20260921004834.2601285-1-skhawaja@google.com/

Thanks.

I have a question regarding the restoration sequencing when module dependencies are involved during a Live Update reboot.

For the standard hardware/driver stack, the sequence seems to naturally follow the kernel's early initcalls and device probing (e.g., IOMMU early hardware handover -> PCI bus topology/BME preservation -> IOMMU domain attach & DMA ownership claim -> VFIO/iommufd cdev binding).

However, if there is a custom kernel module or subsystem (let's call it Module A) that is not part of the standard PCI/IOMMU device probe callback chain, but strictly depends on the fully restored state of PCI, IOMMU, and VFIO/iommufd:

  1. What is the recommended or standardized way in the Live Update architecture to guarantee that Module A's restoration happens after all its underlying dependencies (PCI / IOMMU / VFIO) have completely finished their restore processes?

  2. Is the expectation to rely on LUO (Live Update Orchestrator) phase notification callbacks (e.g., late restore notifiers), Driver Core mechanisms like -EPROBE_DEFER / device_link, or something else?

Any guidance on how cross-subsystem restoration order and async probe dependencies should be handled in the Live Update framework would be greatly appreciated.

Thanks,

Yanjun Zhu


The IOMMU series builds on top of the PCI core support (this series)
and the VFIO series linked further up in the cover letter.

-- 
Best Regards,
Yanjun.Zhu
--------------w02yeVEA7OyGJFRzsrh3hdEm--