All of lore.kernel.org
 help / color / mirror / Atom feed
From: Pranjal Shrivastava <praan@google.com>
To: David Matlack <dmatlack@google.com>
Cc: kexec@lists.infradead.org, linux-doc@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	linux-pci@vger.kernel.org,
	Adithya Jayachandran <ajayachandra@nvidia.com>,
	Alexander Graf <graf@amazon.com>,
	Alex Williamson <alex@shazbot.org>,
	Bjorn Helgaas <bhelgaas@google.com>, Chris Li <chrisl@kernel.org>,
	David Rientjes <rientjes@google.com>,
	Jacob Pan <jacob.pan@linux.microsoft.com>,
	Jason Gunthorpe <jgg@nvidia.com>,
	Jonathan Corbet <corbet@lwn.net>, Josh Hilke <jrhilke@google.com>,
	Leon Romanovsky <leonro@nvidia.com>,
	Lukas Wunner <lukas@wunner.de>, Mike Rapoport <rppt@kernel.org>,
	Parav Pandit <parav@nvidia.com>,
	Pasha Tatashin <pasha.tatashin@soleen.com>,
	Pratyush Yadav <pratyush@kernel.org>,
	Saeed Mahameed <saeedm@nvidia.com>,
	Samiullah Khawaja <skhawaja@google.com>,
	Shuah Khan <skhan@linuxfoundation.org>,
	Vipin Sharma <vipinsh@google.com>, William Tu <witu@nvidia.com>,
	Yi Liu <yi.l.liu@intel.com>
Subject: Re: [PATCH v6 06/12] PCI: liveupdate: Auto-preserve upstream bridges across Live Update
Date: Tue, 9 Jun 2026 11:15:02 +0000	[thread overview]
Message-ID: <aif1tq8TvHuYwUC-@google.com> (raw)
In-Reply-To: <aic1gdmisIT4sqEN@google.com>

On Mon, Jun 08, 2026 at 09:34:57PM +0000, David Matlack wrote:
> On 2026-06-06 10:15 PM, Pranjal Shrivastava wrote:
> > On Fri, May 22, 2026 at 08:24:04PM +0000, David Matlack wrote:
> > > When a PCI device is preserved across a Live Update, all of its upstream
> > > bridges up to the root port must also be preserved. This enables the PCI
> > > core and any drivers bound to the bridges to manage bridges correctly
> > > across a Live Update.
> > > 
> > > Notably, this will be used in subsequent commits to ensure that
> > > preserved devices can continue performing memory transactions without a
> > > disruption or change in routing.
> > > 
> > > To preserve bridges, the PCI core tracks the number of downstream
> > > devices preserved under each bridge using a reference count in struct
> > > pci_dev_ser. This allows a bridge to remain preserved until all its
> > > downstream preserved devices are unpreserved or finish their
> > > participation in the Live Update.
> > > 
> > > Signed-off-by: David Matlack <dmatlack@google.com>
> > > ---
> > >  drivers/pci/liveupdate.c    | 136 +++++++++++++++++++++++++++++++-----
> > >  include/linux/kho/abi/pci.h |   5 +-
> > >  2 files changed, 122 insertions(+), 19 deletions(-)
> > > 
> > 
> > [...]
> > 
> > > +
> > > +#define for_each_pci_dev_in_path(_d, _start, _end) \
> > > +	for ((_d) = (_start); (_d) != (_end); (_d) = (_d)->bus->self)
> > > +
> > > +static void __pci_liveupdate_unpreserve_path(struct pci_ser *ser,
> > > +					     struct pci_dev *start,
> > > +					     struct pci_dev *end)
> > > +{
> > > +	struct pci_dev *dev;
> > > +
> > > +	for_each_pci_dev_in_path(dev, start, end) {
> > > +		if (pci_liveupdate_unpreserve_device(ser, dev))
> > 
> > I might be reading this wrong but are we leaking some upstream devs if 
> > an intermediate node fails?
> > 
> > 			  EP0
> > 			/
> > Assume we have: RC -> B1 -> B2 
> > 				\
> > 				 EP1
> > 
> > and EP0 & EP1 were preserved successfully.
> > 
> > And then we try unpreserving EP1, we follow:
> > 
> > unpreserve EP1 -> unpreserve B2 failed due to a corruption.
> > 
> > This aborts the loop, skipping B1 and RC completely?
> > Their refcounts remain elevated, effectively leaking them as preserved 
> > state permanently? (i.e. if we unpreserve EP0 after this, B1 & RC will
> > still get preserved).
> 
> Yes, but that would only happen if there is some sort of kernel bug or
> silent data corruption. I guess we could proceed with trying to
> unpreserve the bridges upstream. But I opted to log a big warning and
> bail immediately.
> 
> pci_liveupdate_finish_path() has the same behavior BTW.

Fair point. I agree we are in a broken state if we hit this. 

I was originally thinking of a situation where we'd want to keep the
failure localized. For example: unpreserve EP1 fails -> user sees the
warning -> resets EP1 -> retries preserving it later. 

But given the recent discussion/decision that retrieve operations
will no longer be retried, I guess there isn't really a use-case for
retrying anything. It makes sense to just bail here.

> 
> > 
> > > +			return;
> > > +	}
> > > +}
> > > +
> > > +static void pci_liveupdate_unpreserve_path(struct pci_ser *ser,
> > > +					   struct pci_dev *start)
> > > +{
> > > +	__pci_liveupdate_unpreserve_path(ser, start, /*end=*/NULL);
> > > +}
> > > +
> > > +static int pci_liveupdate_preserve_path(struct pci_ser *ser,
> > > +					struct pci_dev *start)
> > > +{
> > > +	struct pci_dev *dev;
> > > +	int ret;
> > > +
> > > +	for_each_pci_dev_in_path(dev, start, NULL) {
> > > +		ret = pci_liveupdate_preserve_device(ser, dev);
> > > +		if (ret) {
> > > +			__pci_liveupdate_unpreserve_path(ser, start, dev);
> > > +			return ret;
> > > +		}
> > > +	}
> > > +
> > > +	return 0;
> > > +}
> > > +
> > >  /**
> > >   * pci_liveupdate_preserve() - Preserve a PCI device across Live Update
> > >   * @dev: The PCI device to preserve.
> > > @@ -321,6 +403,9 @@ static int pci_liveupdate_preserve_device(struct pci_ser *ser, struct pci_dev *d
> > >   * pci_liveupdate_preserve() from their struct liveupdate_file_handler
> > >   * preserve() callback to ensure the outgoing struct pci_ser is already set up.
> > >   *
> > > + * pci_liveupdate_preserve() automatically preserves all bridges upstream of
> > > + * @dev.
> > > + *
> > >   * Returns: 0 on success, <0 on failure.
> > >   */
> > >  int pci_liveupdate_preserve(struct pci_dev *dev)
> > > @@ -336,7 +421,7 @@ int pci_liveupdate_preserve(struct pci_dev *dev)
> > >  	if (IS_ERR(ser))
> > >  		return PTR_ERR(ser);
> > >  
> > > -	return pci_liveupdate_preserve_device(ser, dev);
> > > +	return pci_liveupdate_preserve_path(ser, dev);
> > 
> > Minor nit: I might be too nitpicky here (and it's NOT a strong opinion)
> > but naming it pci_liveupdate_preserve_path_for_dev() reads better to me.
> 
> Noted :). I'll keep the current name for now since that is pretty long,
> but if anyone else votes for it I'm happy to be overridden.

Sounds good.

Thanks,
Praan

  reply	other threads:[~2026-06-09 11:15 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-05-22 20:23 [PATCH v6 00/12] PCI: liveupdate: PCI core support for Live Update David Matlack
2026-05-22 20:23 ` [PATCH v6 01/12] PCI: liveupdate: Set up FLB handler for the PCI core David Matlack
2026-05-22 20:43   ` sashiko-bot
2026-05-22 21:37     ` David Matlack
2026-05-23 11:17       ` Mike Rapoport
2026-05-25 16:50         ` Pratyush Yadav
2026-06-05  5:41   ` Pranjal Shrivastava
2026-06-08 20:51     ` David Matlack
2026-06-09 10:45       ` Pranjal Shrivastava
2026-05-22 20:24 ` [PATCH v6 02/12] PCI: liveupdate: Track outgoing preserved PCI devices David Matlack
2026-05-22 20:54   ` sashiko-bot
2026-05-22 21:28     ` David Matlack
2026-06-05  6:26       ` Pranjal Shrivastava
2026-06-05  6:11   ` Pranjal Shrivastava
2026-05-22 20:24 ` [PATCH v6 03/12] PCI: liveupdate: Track incoming " David Matlack
2026-05-22 21:13   ` sashiko-bot
2026-05-22 21:34     ` David Matlack
2026-06-06 10:08   ` Pranjal Shrivastava
2026-06-08 20:57     ` David Matlack
2026-06-09 10:48       ` Pranjal Shrivastava
2026-05-22 20:24 ` [PATCH v6 04/12] PCI: liveupdate: Document driver binding responsibilities David Matlack
2026-05-25 15:35   ` Pratyush Yadav
2026-06-06 10:20   ` Pranjal Shrivastava
2026-05-22 20:24 ` [PATCH v6 05/12] PCI: liveupdate: Keep bus numbers constant during Live Update David Matlack
2026-05-22 21:08   ` sashiko-bot
2026-05-22 21:31     ` David Matlack
2026-06-06 11:10   ` Pranjal Shrivastava
2026-05-22 20:24 ` [PATCH v6 06/12] PCI: liveupdate: Auto-preserve upstream bridges across " David Matlack
2026-06-06 22:15   ` Pranjal Shrivastava
2026-06-08 21:34     ` David Matlack
2026-06-09 11:15       ` Pranjal Shrivastava [this message]
2026-05-22 20:24 ` [PATCH v6 07/12] PCI: Refactor matching logic for pci_dev_acs_ops David Matlack
2026-06-07 19:01   ` Pranjal Shrivastava
2026-06-08 21:49     ` David Matlack
2026-06-09 10:56       ` Pranjal Shrivastava
2026-05-22 20:24 ` [PATCH v6 08/12] PCI: liveupdate: Inherit ACS flags in incoming preserved devices David Matlack
2026-06-07 20:37   ` Pranjal Shrivastava
2026-06-08 10:49     ` Pranjal Shrivastava
2026-06-08 18:16       ` Jason Gunthorpe
2026-06-09 15:12         ` Pranjal Shrivastava
2026-06-09 15:34           ` Pranjal Shrivastava
2026-06-08 21:56     ` David Matlack
2026-06-09 17:20       ` Pranjal Shrivastava
2026-05-22 20:24 ` [PATCH v6 09/12] PCI: liveupdate: Inherit ARI Forwarding Enable on preserved bridges David Matlack
2026-05-22 21:51   ` sashiko-bot
2026-06-08 11:33   ` Pranjal Shrivastava
2026-06-08 18:19     ` Jason Gunthorpe
2026-05-22 20:24 ` [PATCH v6 10/12] PCI: liveupdate: Freeze preservation status during shutdown David Matlack
2026-06-08 11:47   ` Pranjal Shrivastava
2026-05-22 20:24 ` [PATCH v6 11/12] PCI: liveupdate: Do not disable bus mastering on preserved devices during kexec David Matlack
2026-06-08 11:58   ` Pranjal Shrivastava
2026-05-22 20:24 ` [PATCH v6 12/12] Documentation: PCI: Add documentation for Live Update David Matlack
2026-06-08 12:01   ` Pranjal Shrivastava

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aif1tq8TvHuYwUC-@google.com \
    --to=praan@google.com \
    --cc=ajayachandra@nvidia.com \
    --cc=alex@shazbot.org \
    --cc=bhelgaas@google.com \
    --cc=chrisl@kernel.org \
    --cc=corbet@lwn.net \
    --cc=dmatlack@google.com \
    --cc=graf@amazon.com \
    --cc=jacob.pan@linux.microsoft.com \
    --cc=jgg@nvidia.com \
    --cc=jrhilke@google.com \
    --cc=kexec@lists.infradead.org \
    --cc=leonro@nvidia.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=lukas@wunner.de \
    --cc=parav@nvidia.com \
    --cc=pasha.tatashin@soleen.com \
    --cc=pratyush@kernel.org \
    --cc=rientjes@google.com \
    --cc=rppt@kernel.org \
    --cc=saeedm@nvidia.com \
    --cc=skhan@linuxfoundation.org \
    --cc=skhawaja@google.com \
    --cc=vipinsh@google.com \
    --cc=witu@nvidia.com \
    --cc=yi.l.liu@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.