From: "Fabio M. De Francesco" <fabio.m.de.francesco@linux.intel.com>
To: linux-cxl@vger.kernel.org
Cc: David Hildenbrand <david@kernel.org>,
Oscar Salvador <osalvador@suse.de>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
"Rafael J . Wysocki" <rafael@kernel.org>,
Danilo Krummrich <dakr@kernel.org>,
Davidlohr Bueso <dave@stgolabs.net>,
Jonathan Cameron <jic23@kernel.org>,
Dave Jiang <dave.jiang@intel.com>,
Alison Schofield <alison.schofield@intel.com>,
Vishal Verma <vishal.l.verma@intel.com>,
Ira Weiny <iweiny@kernel.org>, Li Ming <ming.li@zohomail.com>,
Bjorn Helgaas <bhelgaas@google.com>,
Mahesh J Salgaonkar <mahesh@linux.ibm.com>,
Oliver O'Halloran <oohall@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
Lorenzo Stoakes <ljs@kernel.org>,
"Liam R . Howlett" <liam@infradead.org>,
Vlastimil Babka <vbabka@kernel.org>,
Mike Rapoport <rppt@kernel.org>,
Suren Baghdasaryan <surenb@google.com>,
Michal Hocko <mhocko@suse.com>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
driver-core@lists.linux.dev, linux-pci@vger.kernel.org,
linuxppc-dev@lists.ozlabs.org,
"Fabio M. De Francesco" <fabio.m.de.francesco@linux.intel.com>
Subject: [PATCH v2 07/13] cxl/core: Add region disable and enable for a DPort SBR
Date: Tue, 25 Aug 2026 04:26:22 +0200 [thread overview]
Message-ID: <20260825022628.3651434-8-fabio.m.de.francesco@linux.intel.com> (raw)
In-Reply-To: <20260825022628.3651434-1-fabio.m.de.francesco@linux.intel.com>
A Secondary Bus Reset of a CXL Downstream Port removes the downstream
component from the bus, so a CXL region reached through that Port has to
be disabled while the reset runs.
Add cxl_region_disable() to offline memory, invalidate the CPU caches
for its range and release its driver, and cxl_region_enable() to
re-attach the driver afterwards.
Neither function has a caller yet. Both are driven by the reset once the
set of regions routed through the Port is known.
Signed-off-by: Fabio M. De Francesco <fabio.m.de.francesco@linux.intel.com>
---
drivers/cxl/core/core.h | 3 ++
drivers/cxl/core/dport_sbr.c | 79 ++++++++++++++++++++++++++++++++++++
drivers/cxl/core/region.c | 3 +-
3 files changed, 84 insertions(+), 1 deletion(-)
diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index a8b9d7990d32..ac55f9e8160e 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -56,6 +56,9 @@ u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, const struct cxl_memdev *cxlmd,
int devm_cxl_add_dax_region(struct cxl_region *cxlr);
int devm_cxl_add_pmem_region(struct cxl_region *cxlr);
void kill_regions(struct cxl_root_decoder *cxlrd);
+int cxl_region_invalidate_memregion(struct cxl_region *cxlr);
+int cxl_region_disable(struct cxl_region *cxlr);
+void cxl_region_enable(struct cxl_region *cxlr);
struct pci_dev;
void cxl_sbr_recommit_decoders(struct pci_dev *dport_pci,
struct xarray *hdm_state);
diff --git a/drivers/cxl/core/dport_sbr.c b/drivers/cxl/core/dport_sbr.c
index 55c35386d0cb..0f398ab0d3a4 100644
--- a/drivers/cxl/core/dport_sbr.c
+++ b/drivers/cxl/core/dport_sbr.c
@@ -1,11 +1,90 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright(c) 2026 Intel Corporation. */
+#include <linux/memregion.h>
+#include <linux/memory_hotplug.h>
+#include <linux/memory.h>
#include <linux/device.h>
#include <linux/pci.h>
#include <cxl.h>
#include "core.h"
+/*
+ * cxl_region_disable - make a region inactive ahead of a Secondary Bus Reset
+ * @cxlr: region routed through the CXL Downstream Port being reset
+ *
+ * Offline the memory blocks the region owns and unbind its driver. An SBR
+ * zeroes the downstream bus number, so a region left live as System RAM would
+ * be accessed while the device is in reset. On offline failure return the error
+ * so the caller aborts the reset; the memory is never force-removed.
+ *
+ * Context: process context. Offlining and driver unbind sleep and take the
+ * memory hotplug lock, so this cannot run in atomic context.
+ */
+int cxl_region_disable(struct cxl_region *cxlr)
+{
+ struct cxl_region_params *p = &cxlr->params;
+ unsigned long block_size;
+ u64 start, end;
+ int rc;
+
+ /*
+ * Per CXL r4.0 sec 9.13.1 an Interleave Set has a Base HPA and a Size
+ * that are multiples of 256 MB, while a memory block spans up to 2 GB.
+ * A block overlapping either end of the range therefore also covers
+ * memory outside this region, so round the range inward to block
+ * granularity as dax_kmem did when it onlined the range. Offlining a
+ * straddling block would migrate pages that the reset does not affect.
+ */
+ block_size = memory_block_size_bytes();
+ start = ALIGN(p->res->start, block_size);
+ end = ALIGN_DOWN(p->res->end + 1, block_size);
+ if (start >= end) {
+ dev_dbg(&cxlr->dev, "%s: HPA %pr spans no whole memory block, no System RAM to offline\n",
+ __func__, p->res);
+ } else {
+ rc = cxl_offline_memory(start, end - start);
+ if (rc) {
+ dev_warn(&cxlr->dev, "offline System RAM failed before reset: %d\n",
+ rc);
+ return rc;
+ }
+ }
+
+ rc = cxl_region_invalidate_memregion(cxlr);
+ if (rc) {
+ dev_warn(&cxlr->dev, "CPU cache invalidate failed before reset: %d\n",
+ rc);
+ return rc;
+ }
+
+ device_release_driver(&cxlr->dev);
+ dev_dbg(&cxlr->dev, "%s: System RAM offline, region disabled before reset, HPA %pr\n",
+ __func__, p->res);
+
+ return 0;
+}
+
+/*
+ * cxl_region_enable - restore a region after a Secondary Bus Reset
+ * @cxlr: region disabled by cxl_region_disable() before the reset
+ *
+ * Rebind the region driver. The System RAM is left offline; bringing it back
+ * online is a separate administrative step.
+ */
+void cxl_region_enable(struct cxl_region *cxlr)
+{
+ struct cxl_region_params *p = &cxlr->params;
+
+ if (device_attach(&cxlr->dev) < 0) {
+ dev_dbg(&cxlr->dev, "driver re-attach failed after reset\n");
+ return;
+ }
+
+ dev_dbg(&cxlr->dev, "%s: region re-enabled after reset, HPA %pr, IW %d, IG %d\n",
+ __func__, p->res, p->interleave_ways, p->interleave_granularity);
+}
+
/*
* The reset cleared the HDM Decoder registers of every CXL component below
* @dport_pci, so restore them from the settings the driver holds and from
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 1e211542b6b6..fc0bec991a69 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -222,7 +222,7 @@ static struct cxl_region_ref *cxl_rr_load(struct cxl_port *port,
return xa_load(&port->regions, (unsigned long)cxlr);
}
-static int cxl_region_invalidate_memregion(struct cxl_region *cxlr)
+int cxl_region_invalidate_memregion(struct cxl_region *cxlr)
{
if (!cpu_cache_has_invalidate_memregion()) {
if (IS_ENABLED(CONFIG_CXL_REGION_INVALIDATION_TEST)) {
@@ -4273,4 +4273,5 @@ void cxl_region_exit(void)
MODULE_IMPORT_NS("CXL");
MODULE_IMPORT_NS("DEVMEM");
+MODULE_IMPORT_NS("CXL_MHP");
MODULE_ALIAS_CXL(CXL_DEVICE_REGION);
--
2.55.0
next prev parent reply other threads:[~2026-08-25 2:27 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-25 2:26 [PATCH v2 00/13] Make SBR work for CXL Downstream Ports Fabio M. De Francesco
2026-08-25 2:26 ` [PATCH v2 01/13] cxl/pci: Make the HDM and Mem_Enable writes callable from CXL Fabio M. De Francesco
2026-08-25 2:26 ` [PATCH v2 02/13] cxl/hdm: Add function to restore one HDM decoder Fabio M. De Francesco
2026-08-25 2:26 ` [PATCH v2 03/13] cxl/hdm: Add function to restore CXL.mem decode Fabio M. De Francesco
2026-08-25 2:26 ` [PATCH v2 04/13] cxl/hdm: Reprogram the HDM Decoders below a CXL Port Fabio M. De Francesco
2026-08-25 2:26 ` [PATCH v2 05/13] cxl/core: Restore the HDM decoders below DPort Fabio M. De Francesco
2026-08-25 2:26 ` [PATCH v2 06/13] drivers/base/memory: Add cxl_offline_memory() to offline a physical range Fabio M. De Francesco
2026-08-25 2:26 ` Fabio M. De Francesco [this message]
2026-08-25 2:26 ` [PATCH v2 08/13] cxl/core: Collect the regions routed through a DPort Fabio M. De Francesco
2026-08-25 2:26 ` [PATCH v2 09/13] PCI/CXL: Disable and re-enable CXL regions Fabio M. De Francesco
2026-08-26 9:04 ` Richard Cheng
2026-08-25 2:26 ` [PATCH v2 10/13] PCI/CXL: Clear ACS SV across an SBR of a CXL DPort Fabio M. De Francesco
2026-08-26 8:47 ` Richard Cheng
2026-08-25 2:26 ` [PATCH v2 11/13] PCI/DPC: Unbind regions for DPC recovery Fabio M. De Francesco
2026-08-26 8:54 ` Richard Cheng
2026-08-25 2:26 ` [PATCH v2 12/13] PCI/CXL: Add a sysfs entry to unmask SBR Fabio M. De Francesco
2026-08-25 2:26 ` [PATCH v2 13/13] PCI/CXL: Refuse an SBR of a CXL DPort unless authorized Fabio M. De Francesco
2026-08-26 9:21 ` [PATCH v2 00/13] Make SBR work for CXL Downstream Ports Richard Cheng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260825022628.3651434-8-fabio.m.de.francesco@linux.intel.com \
--to=fabio.m.de.francesco@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=alison.schofield@intel.com \
--cc=bhelgaas@google.com \
--cc=dakr@kernel.org \
--cc=dave.jiang@intel.com \
--cc=dave@stgolabs.net \
--cc=david@kernel.org \
--cc=driver-core@lists.linux.dev \
--cc=gregkh@linuxfoundation.org \
--cc=iweiny@kernel.org \
--cc=jic23@kernel.org \
--cc=liam@infradead.org \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-pci@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=ljs@kernel.org \
--cc=mahesh@linux.ibm.com \
--cc=mhocko@suse.com \
--cc=ming.li@zohomail.com \
--cc=oohall@gmail.com \
--cc=osalvador@suse.de \
--cc=rafael@kernel.org \
--cc=rppt@kernel.org \
--cc=surenb@google.com \
--cc=vbabka@kernel.org \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox