[PATCH] pci: Support preferring MMCONFIG for all config space accesses

[PATCH] pci: Support preferring MMCONFIG for all config space accesses

[Josh Triplett]

Even when MMCONFIG is available, PCIe currently defaults to IO-based
accesses for the first 256 bytes of config space.

IO-based accesses are serialized, and can cause substantial scalability
issues when initializing a large number of devices on a system with many
CPUs. (Those often coincide, because many systems have PCIe devices per
CPU.)

Add a `pci=prefer_mmconf` option to use MMCONFIG for all config space
accesses. Also add a Kconfig option (default n) to make that the
default, allowing a boot-time opt out with `pci=no_prefer_mmconf`.

Tested on an AWS c8i.48xlarge system, with 192 CPUs, over five boots
with and five boots without.

Without:
PCI init 266ms
Boot to minimal /init 726ms

With:
PCI init 213ms (~20% faster)
Boot to minimal /init 661ms (~9% faster)

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
---

This has come up a few times, such as in 2017:
https://lore.kernel.org/linux-pci/20170302232104.10136-2-andi@firstfloor.org/

We have even more parallel systems now than we did then, and the
performance hit from the serialized IO path hurts.

This *should* be safe everywhere. It seems exceedingly unlikely that any
systems need this. Despite that, this patch makes it `default n` out of
an abundance of caution.

I expect this to have even more impact on larger systems. I'd be
thrilled to test this on a 384-way system, if I had access to one. If
anyone at AWS would like to approve quota increase case ID
179306079200327 so I can test on a c8i.96xlarge, I'd be happy to test
that. ;)


 .../admin-guide/kernel-parameters.txt         |  5 +++
 arch/x86/Kconfig                              | 24 +++++++++++
 arch/x86/pci/common.c                         | 41 +++++++++++++++++--
 3 files changed, 67 insertions(+), 3 deletions(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index d0f174288365..64a138ea8f54 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -5018,6 +5018,11 @@ Kernel parameters
 				disable the use of PCIE advanced error reporting.
 		nodomains	[PCI] Disable support for multiple PCI
 				root domains (aka PCI segments, in ACPI-speak).
+		prefer_mmconf	[X86] Prefer use of MMCONFIG even for base (<256)
+				config space accesses.
+		no_prefer_mmconf	[X86] Don't prefer use of MMCONFIG for
+				base (<256) config space accesses, even if
+				CONFIG_PCI_MMCONFIG_PREFERRED=y.
 		nommconf	[X86] Disable use of MMCONFIG for PCI
 				Configuration
 		check_enable_amd_mmconf [X86] check for and enable
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index bdad90f210e4..e8f2d02562ed 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2888,6 +2888,30 @@ config PCI_MMCONFIG
 
 	  Say Y otherwise.
 
+config PCI_MMCONFIG_PREFERRED
+	bool "Prefer mmconfig for all accesses"
+	depends on PCI_MMCONFIG
+	help
+	  Use memory-mapped accesses for all PCI config space accesses where
+	  supported.
+
+	  Historically, Linux used legacy direct accesses for the first 256
+	  bytes of each device's configuration space, even if memory-mapped
+	  accesses were supported. Legacy accesses are serialized across all
+	  CPUs, so they can become a bottleneck on systems with many CPUs.
+	  Enabling this option substantially improves the scalability of PCI
+	  config space accesses.
+
+	  In the unlikely case that enabling this configuration option causes
+	  problems, you can switch it off with `pci=no_prefer_mmconf`.
+
+	  Even if this option is disabled, it can be set at boot time with
+	  `pci=prefer_mmconf`; this option just sets the default.
+
+	  You should say Y unless you have a very old system that doesn't
+	  support this. If you have a system that requires saying N here,
+	  please report it.
+
 config PCI_OLPC
 	def_bool y
 	depends on PCI && OLPC && (PCI_GOOLPC || PCI_GOANY)
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index 63105b29fbf5..13e547f88d75 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -26,6 +26,13 @@ unsigned int pci_probe = PCI_PROBE_BIOS | PCI_PROBE_CONF1 | PCI_PROBE_CONF2 |
 
 static int pci_bf_sort;
 int pci_routeirq;
+#ifdef CONFIG_PCI_MMCONFIG
+#ifdef CONFIG_PCI_MMCONFIG_PREFERRED
+static int pci_prefer_mmconf = 1;
+#else
+static int pci_prefer_mmconf;
+#endif
+#endif /* #ifdef CONFIG_PCI_MMCONFIG */
 int noioapicquirk;
 #ifdef CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS
 int noioapicreroute = 0;
@@ -40,9 +47,20 @@ const struct pci_raw_ops *__read_mostly raw_pci_ext_ops;
 int raw_pci_read(unsigned int domain, unsigned int bus, unsigned int devfn,
 						int reg, int len, u32 *val)
 {
+	bool tried_mmconf = false;
+
+#ifdef CONFIG_PCI_MMCONFIG
+	if (pci_prefer_mmconf && raw_pci_ext_ops == &pci_mmcfg) {
+		int ret = raw_pci_ext_ops->read(domain, bus, devfn, reg, len, val);
+
+		if (ret != -EINVAL)
+			return ret;
+		tried_mmconf = true;
+	}
+#endif
 	if (domain == 0 && reg < 256 && raw_pci_ops)
 		return raw_pci_ops->read(domain, bus, devfn, reg, len, val);
-	if (raw_pci_ext_ops)
+	if (!tried_mmconf && raw_pci_ext_ops)
 		return raw_pci_ext_ops->read(domain, bus, devfn, reg, len, val);
 	return -EINVAL;
 }
@@ -50,9 +68,20 @@ int raw_pci_read(unsigned int domain, unsigned int bus, unsigned int devfn,
 int raw_pci_write(unsigned int domain, unsigned int bus, unsigned int devfn,
 						int reg, int len, u32 val)
 {
+	bool tried_mmconf = false;
+
+#ifdef CONFIG_PCI_MMCONFIG
+	if (pci_prefer_mmconf && raw_pci_ext_ops == &pci_mmcfg) {
+		int ret = raw_pci_ext_ops->write(domain, bus, devfn, reg, len, val);
+
+		if (ret != -EINVAL)
+			return ret;
+		tried_mmconf = true;
+	}
+#endif
 	if (domain == 0 && reg < 256 && raw_pci_ops)
 		return raw_pci_ops->write(domain, bus, devfn, reg, len, val);
-	if (raw_pci_ext_ops)
+	if (!tried_mmconf && raw_pci_ext_ops)
 		return raw_pci_ext_ops->write(domain, bus, devfn, reg, len, val);
 	return -EINVAL;
 }
@@ -551,7 +580,13 @@ char *__init pcibios_setup(char *str)
 	}
 #endif
 #ifdef CONFIG_PCI_MMCONFIG
-	else if (!strcmp(str, "nommconf")) {
+	else if (!strcmp(str, "prefer_mmconf")) {
+		pci_prefer_mmconf = 1;
+		return NULL;
+	} else if (!strcmp(str, "no_prefer_mmconf")) {
+		pci_prefer_mmconf = 0;
+		return NULL;
+	} else if (!strcmp(str, "nommconf")) {
 		pci_probe &= ~PCI_PROBE_MMCONF;
 		return NULL;
 	}
-- 
2.55.0

Re: [PATCH] pci: Support preferring MMCONFIG for all config space accesses

[sashiko-bot]

> Even when MMCONFIG is available, PCIe currently defaults to IO-based
> accesses for the first 256 bytes of config space.
> 
> IO-based accesses are serialized, and can cause substantial scalability
> issues when initializing a large number of devices on a system with many
> CPUs. (Those often coincide, because many systems have PCIe devices per
> CPU.)
> 
> Add a `pci=prefer_mmconf` option to use MMCONFIG for all config space
> accesses. Also add a Kconfig option (default n) to make that the
> default, allowing a boot-time opt out with `pci=no_prefer_mmconf`.
> 
> Tested on an AWS c8i.48xlarge system, with 192 CPUs, over five boots
> with and five boots without.
> 
> Without:
> PCI init 266ms
> Boot to minimal /init 726ms
> 
> With:
> PCI init 213ms (~20% faster)
> Boot to minimal /init 661ms (~9% faster)
> 
> Signed-off-by: Josh Triplett <josh@joshtriplett.org>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/a7d2b402784a409f5564c1fdce77a41e09f29280.1793087687.git.josh@joshtriplett.org?part=1