* [PATCH v2 0/2] PCI: Unify domain emulation
@ 2025-10-24 22:46 Dan Williams
2025-10-24 22:46 ` [PATCH v2 1/2] PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms Dan Williams
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Dan Williams @ 2025-10-24 22:46 UTC (permalink / raw)
To: bhelgaas
Cc: linux-pci, jonathan.derrick, lpieralisi, kwilczynski, mani, robh,
Dexuan Cui, Haiyang Zhang, K. Y. Srinivasan,
Manivannan Sadhasivam, Michael Kelley, Nirmal Patel,
Suzuki K Poulose, Szymon Durawa, Wei Liu
Changes since v1 [1]:
- Rebase on v6.18-rc2
- Support callers supplying both a hint and a range for the Hyper-V hint
+ fallback case (Michael)
- Add comment explaining domain number 0 vs Gen1 VMs, and domain values
greater than U16_MAX concerns (Michael)
- Leave the VMD status quo comment about requesting domain numbers >
U16_MAX
[1]: http://lore.kernel.org/20250716160835.680486-1-dan.j.williams@intel.com
The PCI/TSM effort created a sample driver to test ABI flows
(samples/devsec/ [2]). Suzuki observed that it only worked on x86 due to
its dependency on CONFIG_PCI_DOMAINS_GENERIC=n. I.e. an unfortunate
restriction for what should be an architecture agnostic test framework.
Introduce a new pci_bus_find_emul_domain_nr() helper that all "soft"
host-bridge drivers can share and hide the CONFIG_PCI_DOMAINS_GENERIC
details behind that helper.
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm.git/commit/?id=0e16ce0b9c64
Dan Williams (2):
PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms
PCI: vmd: Switch to pci_bus_find_emul_domain_nr()
include/linux/pci.h | 7 ++++
drivers/pci/controller/pci-hyperv.c | 62 +++++------------------------
drivers/pci/controller/vmd.c | 40 ++++++++-----------
drivers/pci/pci.c | 24 ++++++++++-
drivers/pci/probe.c | 8 +++-
5 files changed, 63 insertions(+), 78 deletions(-)
base-commit: 211ddde0823f1442e4ad052a2f30f050145ccada
--
2.51.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 1/2] PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms
2025-10-24 22:46 [PATCH v2 0/2] PCI: Unify domain emulation Dan Williams
@ 2025-10-24 22:46 ` Dan Williams
2025-10-25 19:53 ` Michael Kelley
2025-10-27 20:08 ` Dave Jiang
2025-10-24 22:46 ` [PATCH v2 2/2] PCI: vmd: Switch to pci_bus_find_emul_domain_nr() Dan Williams
2025-10-28 17:39 ` [PATCH v2 0/2] PCI: Unify domain emulation Bjorn Helgaas
2 siblings, 2 replies; 8+ messages in thread
From: Dan Williams @ 2025-10-24 22:46 UTC (permalink / raw)
To: bhelgaas
Cc: linux-pci, jonathan.derrick, lpieralisi, kwilczynski, mani, robh,
Suzuki K Poulose, Manivannan Sadhasivam, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Michael Kelley
The ability to emulate a host bridge is useful not only for hardware PCI
controllers like CONFIG_VMD, or virtual PCI controllers like
CONFIG_PCI_HYPERV, but also for test and development scenarios like
CONFIG_SAMPLES_DEVSEC [1].
One stumbling block for defining CONFIG_SAMPLES_DEVSEC, a sample
implementation of a platform TSM for PCI Device Security, is the need to
accommodate PCI_DOMAINS_GENERIC architectures alongside x86 [2].
In support of supplementing the existing CONFIG_PCI_BRIDGE_EMUL
infrastructure for host bridges:
* Introduce pci_bus_find_emul_domain_nr() as a common way to find a free
PCI domain number whether that is to reuse the existing dynamic
allocation code in the !ACPI case, or to assign an unused domain above
the last ACPI segment.
* Convert pci-hyperv to the new allocator so that the PCI core can
unconditionally assume that bridge->domain_nr != PCI_DOMAIN_NR_NOT_SET
is the dynamically allocated case.
A follow on patch can also convert vmd to the new scheme. Currently vmd
is limited to CONFIG_PCI_DOMAINS_GENERIC=n (x86) so, unlike pci-hyperv,
it does not immediately conflict with this new
pci_bus_find_emul_domain_nr() mechanism.
Link: http://lore.kernel.org/174107249038.1288555.12362100502109498455.stgit@dwillia2-xfh.jf.intel.com [1]
Reported-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Closes: http://lore.kernel.org/20250311144601.145736-3-suzuki.poulose@arm.com
Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>
Cc: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Cc: Rob Herring <robh@kernel.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Wei Liu <wei.liu@kernel.org>
Cc: Dexuan Cui <decui@microsoft.com>
Tested-by: Suzuki K Poulose <suzuki.poulose@arm.com>
[michael: maintain compatibility with userspace that expects 16-bit ids]
Cc: Michael Kelley <mhklinux@outlook.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
include/linux/pci.h | 7 ++++
drivers/pci/controller/pci-hyperv.c | 62 +++++------------------------
drivers/pci/pci.c | 24 ++++++++++-
drivers/pci/probe.c | 8 +++-
4 files changed, 46 insertions(+), 55 deletions(-)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index d1fdf81fbe1e..1ef1535802b0 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1956,10 +1956,17 @@ DEFINE_GUARD(pci_dev, struct pci_dev *, pci_dev_lock(_T), pci_dev_unlock(_T))
*/
#ifdef CONFIG_PCI_DOMAINS
extern int pci_domains_supported;
+int pci_bus_find_emul_domain_nr(u32 hint, u32 min, u32 max);
+void pci_bus_release_emul_domain_nr(int domain_nr);
#else
enum { pci_domains_supported = 0 };
static inline int pci_domain_nr(struct pci_bus *bus) { return 0; }
static inline int pci_proc_domain(struct pci_bus *bus) { return 0; }
+static inline int pci_bus_find_emul_domain_nr(u32 hint, u32 min, u32 max)
+{
+ return 0;
+}
+static inline void pci_bus_release_emul_domain_nr(int domain_nr) { }
#endif /* CONFIG_PCI_DOMAINS */
/*
diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 146b43981b27..64f68efaf547 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -3696,48 +3696,6 @@ static int hv_send_resources_released(struct hv_device *hdev)
return 0;
}
-#define HVPCI_DOM_MAP_SIZE (64 * 1024)
-static DECLARE_BITMAP(hvpci_dom_map, HVPCI_DOM_MAP_SIZE);
-
-/*
- * PCI domain number 0 is used by emulated devices on Gen1 VMs, so define 0
- * as invalid for passthrough PCI devices of this driver.
- */
-#define HVPCI_DOM_INVALID 0
-
-/**
- * hv_get_dom_num() - Get a valid PCI domain number
- * Check if the PCI domain number is in use, and return another number if
- * it is in use.
- *
- * @dom: Requested domain number
- *
- * return: domain number on success, HVPCI_DOM_INVALID on failure
- */
-static u16 hv_get_dom_num(u16 dom)
-{
- unsigned int i;
-
- if (test_and_set_bit(dom, hvpci_dom_map) == 0)
- return dom;
-
- for_each_clear_bit(i, hvpci_dom_map, HVPCI_DOM_MAP_SIZE) {
- if (test_and_set_bit(i, hvpci_dom_map) == 0)
- return i;
- }
-
- return HVPCI_DOM_INVALID;
-}
-
-/**
- * hv_put_dom_num() - Mark the PCI domain number as free
- * @dom: Domain number to be freed
- */
-static void hv_put_dom_num(u16 dom)
-{
- clear_bit(dom, hvpci_dom_map);
-}
-
/**
* hv_pci_probe() - New VMBus channel probe, for a root PCI bus
* @hdev: VMBus's tracking struct for this root PCI bus
@@ -3750,9 +3708,9 @@ static int hv_pci_probe(struct hv_device *hdev,
{
struct pci_host_bridge *bridge;
struct hv_pcibus_device *hbus;
- u16 dom_req, dom;
+ int ret, dom;
+ u16 dom_req;
char *name;
- int ret;
bridge = devm_pci_alloc_host_bridge(&hdev->device, 0);
if (!bridge)
@@ -3779,11 +3737,14 @@ static int hv_pci_probe(struct hv_device *hdev,
* PCI bus (which is actually emulated by the hypervisor) is domain 0.
* (2) There will be no overlap between domains (after fixing possible
* collisions) in the same VM.
+ *
+ * Because Gen1 VMs use domain 0, don't allow picking domain 0 here,
+ * even if bytes 4 and 5 of the instance GUID are both zero. For wider
+ * userspace compatibility, limit the domain id to a 16-bit value.
*/
dom_req = hdev->dev_instance.b[5] << 8 | hdev->dev_instance.b[4];
- dom = hv_get_dom_num(dom_req);
-
- if (dom == HVPCI_DOM_INVALID) {
+ dom = pci_bus_find_emul_domain_nr(dom_req, 1, U16_MAX);
+ if (dom < 0) {
dev_err(&hdev->device,
"Unable to use dom# 0x%x or other numbers", dom_req);
ret = -EINVAL;
@@ -3917,7 +3878,7 @@ static int hv_pci_probe(struct hv_device *hdev,
destroy_wq:
destroy_workqueue(hbus->wq);
free_dom:
- hv_put_dom_num(hbus->bridge->domain_nr);
+ pci_bus_release_emul_domain_nr(hbus->bridge->domain_nr);
free_bus:
kfree(hbus);
return ret;
@@ -4042,8 +4003,6 @@ static void hv_pci_remove(struct hv_device *hdev)
irq_domain_remove(hbus->irq_domain);
irq_domain_free_fwnode(hbus->fwnode);
- hv_put_dom_num(hbus->bridge->domain_nr);
-
kfree(hbus);
}
@@ -4217,9 +4176,6 @@ static int __init init_hv_pci_drv(void)
if (ret)
return ret;
- /* Set the invalid domain number's bit, so it will not be used */
- set_bit(HVPCI_DOM_INVALID, hvpci_dom_map);
-
/* Initialize PCI block r/w interface */
hvpci_block_ops.read_block = hv_read_config_block;
hvpci_block_ops.write_block = hv_write_config_block;
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index b14dd064006c..64aed8705e91 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -6656,9 +6656,31 @@ static void pci_no_domains(void)
#endif
}
+#ifdef CONFIG_PCI_DOMAINS
+static DEFINE_IDA(pci_domain_nr_dynamic_ida);
+
+/**
+ * pci_bus_find_emul_domain_nr() - allocate a PCI domain number per constraints
+ * @hint: desired domain, 0 if any id in the range of @min to @max is acceptable
+ * @min: minimum allowable domain
+ * @max: maximum allowable domain, no ids higher than INT_MAX will be returned
+ */
+int pci_bus_find_emul_domain_nr(u32 hint, u32 min, u32 max)
+{
+ return ida_alloc_range(&pci_domain_nr_dynamic_ida, max(hint, min), max,
+ GFP_KERNEL);
+}
+EXPORT_SYMBOL_GPL(pci_bus_find_emul_domain_nr);
+
+void pci_bus_release_emul_domain_nr(int domain_nr)
+{
+ ida_free(&pci_domain_nr_dynamic_ida, domain_nr);
+}
+EXPORT_SYMBOL_GPL(pci_bus_release_emul_domain_nr);
+#endif
+
#ifdef CONFIG_PCI_DOMAINS_GENERIC
static DEFINE_IDA(pci_domain_nr_static_ida);
-static DEFINE_IDA(pci_domain_nr_dynamic_ida);
static void of_pci_reserve_static_domain_nr(void)
{
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 0ce98e18b5a8..5e101ced00a5 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -650,6 +650,11 @@ static void pci_release_host_bridge_dev(struct device *dev)
pci_free_resource_list(&bridge->windows);
pci_free_resource_list(&bridge->dma_ranges);
+
+ /* Host bridges only have domain_nr set in the emulation case */
+ if (bridge->domain_nr != PCI_DOMAIN_NR_NOT_SET)
+ pci_bus_release_emul_domain_nr(bridge->domain_nr);
+
kfree(bridge);
}
@@ -1130,7 +1135,8 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
device_del(&bridge->dev);
free:
#ifdef CONFIG_PCI_DOMAINS_GENERIC
- pci_bus_release_domain_nr(parent, bus->domain_nr);
+ if (bridge->domain_nr == PCI_DOMAIN_NR_NOT_SET)
+ pci_bus_release_domain_nr(parent, bus->domain_nr);
#endif
if (bus_registered)
put_device(&bus->dev);
--
2.51.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 2/2] PCI: vmd: Switch to pci_bus_find_emul_domain_nr()
2025-10-24 22:46 [PATCH v2 0/2] PCI: Unify domain emulation Dan Williams
2025-10-24 22:46 ` [PATCH v2 1/2] PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms Dan Williams
@ 2025-10-24 22:46 ` Dan Williams
2025-10-27 20:08 ` Dave Jiang
2025-10-28 17:39 ` [PATCH v2 0/2] PCI: Unify domain emulation Bjorn Helgaas
2 siblings, 1 reply; 8+ messages in thread
From: Dan Williams @ 2025-10-24 22:46 UTC (permalink / raw)
To: bhelgaas
Cc: linux-pci, jonathan.derrick, lpieralisi, kwilczynski, mani, robh,
Szymon Durawa, Nirmal Patel
The new common domain number allocator can replace the custom allocator
in VMD.
Beyond some code reuse benefits it does close a potential race whereby
vmd_find_free_domain() collides with new PCI buses coming online with a
conflicting domain number. Such a race has not been observed in
practice, hence not tagging this change as a fix.
As VMD uses pci_create_root_bus() rather than pci_alloc_host_bridge() +
pci_scan_root_bus_bridge() it has no chance to set ->domain_nr in the
bridge so needs to manage freeing the domain number on its own.
Cc: Szymon Durawa <szymon.durawa@linux.intel.com>
Cc: Nirmal Patel <nirmal.patel@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/pci/controller/vmd.c | 40 +++++++++++++++---------------------
1 file changed, 17 insertions(+), 23 deletions(-)
diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index b4b62b9ccc45..03b5920138f0 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -578,22 +578,6 @@ static void vmd_detach_resources(struct vmd_dev *vmd)
vmd->dev->resource[VMD_MEMBAR2].child = NULL;
}
-/*
- * VMD domains start at 0x10000 to not clash with ACPI _SEG domains.
- * Per ACPI r6.0, sec 6.5.6, _SEG returns an integer, of which the lower
- * 16 bits are the PCI Segment Group (domain) number. Other bits are
- * currently reserved.
- */
-static int vmd_find_free_domain(void)
-{
- int domain = 0xffff;
- struct pci_bus *bus = NULL;
-
- while ((bus = pci_find_next_bus(bus)) != NULL)
- domain = max_t(int, domain, pci_domain_nr(bus));
- return domain + 1;
-}
-
static int vmd_get_phys_offsets(struct vmd_dev *vmd, bool native_hint,
resource_size_t *offset1,
resource_size_t *offset2)
@@ -878,13 +862,6 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
.parent = res,
};
- sd->vmd_dev = vmd->dev;
- sd->domain = vmd_find_free_domain();
- if (sd->domain < 0)
- return sd->domain;
-
- sd->node = pcibus_to_node(vmd->dev->bus);
-
/*
* Currently MSI remapping must be enabled in guest passthrough mode
* due to some missing interrupt remapping plumbing. This is probably
@@ -910,9 +887,24 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
pci_add_resource_offset(&resources, &vmd->resources[1], offset[0]);
pci_add_resource_offset(&resources, &vmd->resources[2], offset[1]);
+ sd->vmd_dev = vmd->dev;
+
+ /*
+ * Emulated domains start at 0x10000 to not clash with ACPI _SEG
+ * domains. Per ACPI r6.0, sec 6.5.6, _SEG returns an integer, of
+ * which the lower 16 bits are the PCI Segment Group (domain) number.
+ * Other bits are currently reserved.
+ */
+ sd->domain = pci_bus_find_emul_domain_nr(0, 0x10000, INT_MAX);
+ if (sd->domain < 0)
+ return sd->domain;
+
+ sd->node = pcibus_to_node(vmd->dev->bus);
+
vmd->bus = pci_create_root_bus(&vmd->dev->dev, vmd->busn_start,
&vmd_ops, sd, &resources);
if (!vmd->bus) {
+ pci_bus_release_emul_domain_nr(sd->domain);
pci_free_resource_list(&resources);
vmd_remove_irq_domain(vmd);
return -ENODEV;
@@ -1005,6 +997,7 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
return -ENOMEM;
vmd->dev = dev;
+ vmd->sysdata.domain = PCI_DOMAIN_NR_NOT_SET;
vmd->instance = ida_alloc(&vmd_instance_ida, GFP_KERNEL);
if (vmd->instance < 0)
return vmd->instance;
@@ -1070,6 +1063,7 @@ static void vmd_remove(struct pci_dev *dev)
vmd_detach_resources(vmd);
vmd_remove_irq_domain(vmd);
ida_free(&vmd_instance_ida, vmd->instance);
+ pci_bus_release_emul_domain_nr(vmd->sysdata.domain);
}
static void vmd_shutdown(struct pci_dev *dev)
--
2.51.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* RE: [PATCH v2 1/2] PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms
2025-10-24 22:46 ` [PATCH v2 1/2] PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms Dan Williams
@ 2025-10-25 19:53 ` Michael Kelley
2025-10-27 23:45 ` dan.j.williams
2025-10-27 20:08 ` Dave Jiang
1 sibling, 1 reply; 8+ messages in thread
From: Michael Kelley @ 2025-10-25 19:53 UTC (permalink / raw)
To: Dan Williams, bhelgaas@google.com
Cc: linux-pci@vger.kernel.org, jonathan.derrick@linux.dev,
lpieralisi@kernel.org, kwilczynski@kernel.org, mani@kernel.org,
robh@kernel.org, Suzuki K Poulose, Manivannan Sadhasivam,
K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui
From: Dan Williams <dan.j.williams@intel.com> Sent: Friday, October 24, 2025 3:46 PM
>
> The ability to emulate a host bridge is useful not only for hardware PCI
> controllers like CONFIG_VMD, or virtual PCI controllers like
> CONFIG_PCI_HYPERV, but also for test and development scenarios like
> CONFIG_SAMPLES_DEVSEC [1].
>
> One stumbling block for defining CONFIG_SAMPLES_DEVSEC, a sample
> implementation of a platform TSM for PCI Device Security, is the need to
> accommodate PCI_DOMAINS_GENERIC architectures alongside x86 [2].
There's not a [2] tag anywhere below. Presumably it should be the "Closes:"
link?
>
> In support of supplementing the existing CONFIG_PCI_BRIDGE_EMUL
> infrastructure for host bridges:
>
> * Introduce pci_bus_find_emul_domain_nr() as a common way to find a free
> PCI domain number whether that is to reuse the existing dynamic
> allocation code in the !ACPI case, or to assign an unused domain above
> the last ACPI segment.
>
> * Convert pci-hyperv to the new allocator so that the PCI core can
> unconditionally assume that bridge->domain_nr != PCI_DOMAIN_NR_NOT_SET
> is the dynamically allocated case.
>
> A follow on patch can also convert vmd to the new scheme. Currently vmd
> is limited to CONFIG_PCI_DOMAINS_GENERIC=n (x86) so, unlike pci-hyperv,
> it does not immediately conflict with this new
> pci_bus_find_emul_domain_nr() mechanism.
>
> Link: https://lore.kernel.org/all/174107249038.1288555.12362100502109498455.stgit@dwillia2-xfh.jf.intel.com [1]
> Reported-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> Closes: https://lore.kernel.org/all/20250311144601.145736-3-suzuki.poulose@arm.com
> Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>
> Cc: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: "K. Y. Srinivasan" <kys@microsoft.com>
> Cc: Haiyang Zhang <haiyangz@microsoft.com>
> Cc: Wei Liu <wei.liu@kernel.org>
> Cc: Dexuan Cui <decui@microsoft.com>
> Tested-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> [michael: maintain compatibility with userspace that expects 16-bit ids]
Is the above line spurious? It doesn't seem to belong here.
> Cc: Michael Kelley <mhklinux@outlook.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Tested on x86 and arm64 VMs in the Azure public cloud with
multiple Hyper-V virtual PCI devices in each VM. So covered
both the "CONFIG_PCI_DOMAINS_GENERIC=n" and "=y" cases.
Did manual unbind/rebind of vPCI devices multiple times so the
domain numbers would be freed and reallocated. All was good.
I did not go to the trouble of simulating a domain number
collision.
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
> ---
> include/linux/pci.h | 7 ++++
> drivers/pci/controller/pci-hyperv.c | 62 +++++------------------------
> drivers/pci/pci.c | 24 ++++++++++-
> drivers/pci/probe.c | 8 +++-
> 4 files changed, 46 insertions(+), 55 deletions(-)
>
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index d1fdf81fbe1e..1ef1535802b0 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1956,10 +1956,17 @@ DEFINE_GUARD(pci_dev, struct pci_dev *, pci_dev_lock(_T), pci_dev_unlock(_T))
> */
> #ifdef CONFIG_PCI_DOMAINS
> extern int pci_domains_supported;
> +int pci_bus_find_emul_domain_nr(u32 hint, u32 min, u32 max);
> +void pci_bus_release_emul_domain_nr(int domain_nr);
> #else
> enum { pci_domains_supported = 0 };
> static inline int pci_domain_nr(struct pci_bus *bus) { return 0; }
> static inline int pci_proc_domain(struct pci_bus *bus) { return 0; }
> +static inline int pci_bus_find_emul_domain_nr(u32 hint, u32 min, u32 max)
> +{
> + return 0;
> +}
> +static inline void pci_bus_release_emul_domain_nr(int domain_nr) { }
> #endif /* CONFIG_PCI_DOMAINS */
>
> /*
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index 146b43981b27..64f68efaf547 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -3696,48 +3696,6 @@ static int hv_send_resources_released(struct hv_device *hdev)
> return 0;
> }
>
> -#define HVPCI_DOM_MAP_SIZE (64 * 1024)
> -static DECLARE_BITMAP(hvpci_dom_map, HVPCI_DOM_MAP_SIZE);
> -
> -/*
> - * PCI domain number 0 is used by emulated devices on Gen1 VMs, so define 0
> - * as invalid for passthrough PCI devices of this driver.
> - */
> -#define HVPCI_DOM_INVALID 0
> -
> -/**
> - * hv_get_dom_num() - Get a valid PCI domain number
> - * Check if the PCI domain number is in use, and return another number if
> - * it is in use.
> - *
> - * @dom: Requested domain number
> - *
> - * return: domain number on success, HVPCI_DOM_INVALID on failure
> - */
> -static u16 hv_get_dom_num(u16 dom)
> -{
> - unsigned int i;
> -
> - if (test_and_set_bit(dom, hvpci_dom_map) == 0)
> - return dom;
> -
> - for_each_clear_bit(i, hvpci_dom_map, HVPCI_DOM_MAP_SIZE) {
> - if (test_and_set_bit(i, hvpci_dom_map) == 0)
> - return i;
> - }
> -
> - return HVPCI_DOM_INVALID;
> -}
> -
> -/**
> - * hv_put_dom_num() - Mark the PCI domain number as free
> - * @dom: Domain number to be freed
> - */
> -static void hv_put_dom_num(u16 dom)
> -{
> - clear_bit(dom, hvpci_dom_map);
> -}
> -
> /**
> * hv_pci_probe() - New VMBus channel probe, for a root PCI bus
> * @hdev: VMBus's tracking struct for this root PCI bus
> @@ -3750,9 +3708,9 @@ static int hv_pci_probe(struct hv_device *hdev,
> {
> struct pci_host_bridge *bridge;
> struct hv_pcibus_device *hbus;
> - u16 dom_req, dom;
> + int ret, dom;
> + u16 dom_req;
> char *name;
> - int ret;
>
> bridge = devm_pci_alloc_host_bridge(&hdev->device, 0);
> if (!bridge)
> @@ -3779,11 +3737,14 @@ static int hv_pci_probe(struct hv_device *hdev,
> * PCI bus (which is actually emulated by the hypervisor) is domain 0.
> * (2) There will be no overlap between domains (after fixing possible
> * collisions) in the same VM.
> + *
> + * Because Gen1 VMs use domain 0, don't allow picking domain 0 here,
> + * even if bytes 4 and 5 of the instance GUID are both zero. For wider
> + * userspace compatibility, limit the domain id to a 16-bit value.
> */
> dom_req = hdev->dev_instance.b[5] << 8 | hdev->dev_instance.b[4];
> - dom = hv_get_dom_num(dom_req);
> -
> - if (dom == HVPCI_DOM_INVALID) {
> + dom = pci_bus_find_emul_domain_nr(dom_req, 1, U16_MAX);
> + if (dom < 0) {
> dev_err(&hdev->device,
> "Unable to use dom# 0x%x or other numbers", dom_req);
> ret = -EINVAL;
> @@ -3917,7 +3878,7 @@ static int hv_pci_probe(struct hv_device *hdev,
> destroy_wq:
> destroy_workqueue(hbus->wq);
> free_dom:
> - hv_put_dom_num(hbus->bridge->domain_nr);
> + pci_bus_release_emul_domain_nr(hbus->bridge->domain_nr);
> free_bus:
> kfree(hbus);
> return ret;
> @@ -4042,8 +4003,6 @@ static void hv_pci_remove(struct hv_device *hdev)
> irq_domain_remove(hbus->irq_domain);
> irq_domain_free_fwnode(hbus->fwnode);
>
> - hv_put_dom_num(hbus->bridge->domain_nr);
> -
> kfree(hbus);
> }
>
> @@ -4217,9 +4176,6 @@ static int __init init_hv_pci_drv(void)
> if (ret)
> return ret;
>
> - /* Set the invalid domain number's bit, so it will not be used */
> - set_bit(HVPCI_DOM_INVALID, hvpci_dom_map);
> -
> /* Initialize PCI block r/w interface */
> hvpci_block_ops.read_block = hv_read_config_block;
> hvpci_block_ops.write_block = hv_write_config_block;
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index b14dd064006c..64aed8705e91 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -6656,9 +6656,31 @@ static void pci_no_domains(void)
> #endif
> }
>
> +#ifdef CONFIG_PCI_DOMAINS
> +static DEFINE_IDA(pci_domain_nr_dynamic_ida);
> +
> +/**
> + * pci_bus_find_emul_domain_nr() - allocate a PCI domain number per constraints
> + * @hint: desired domain, 0 if any id in the range of @min to @max is acceptable
> + * @min: minimum allowable domain
> + * @max: maximum allowable domain, no ids higher than INT_MAX will be returned
> + */
> +int pci_bus_find_emul_domain_nr(u32 hint, u32 min, u32 max)
> +{
> + return ida_alloc_range(&pci_domain_nr_dynamic_ida, max(hint, min), max,
> + GFP_KERNEL);
> +}
> +EXPORT_SYMBOL_GPL(pci_bus_find_emul_domain_nr);
> +
> +void pci_bus_release_emul_domain_nr(int domain_nr)
> +{
> + ida_free(&pci_domain_nr_dynamic_ida, domain_nr);
> +}
> +EXPORT_SYMBOL_GPL(pci_bus_release_emul_domain_nr);
> +#endif
> +
> #ifdef CONFIG_PCI_DOMAINS_GENERIC
> static DEFINE_IDA(pci_domain_nr_static_ida);
> -static DEFINE_IDA(pci_domain_nr_dynamic_ida);
>
> static void of_pci_reserve_static_domain_nr(void)
> {
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 0ce98e18b5a8..5e101ced00a5 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -650,6 +650,11 @@ static void pci_release_host_bridge_dev(struct device *dev)
>
> pci_free_resource_list(&bridge->windows);
> pci_free_resource_list(&bridge->dma_ranges);
> +
> + /* Host bridges only have domain_nr set in the emulation case */
> + if (bridge->domain_nr != PCI_DOMAIN_NR_NOT_SET)
> + pci_bus_release_emul_domain_nr(bridge->domain_nr);
> +
> kfree(bridge);
> }
>
> @@ -1130,7 +1135,8 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
> device_del(&bridge->dev);
> free:
> #ifdef CONFIG_PCI_DOMAINS_GENERIC
> - pci_bus_release_domain_nr(parent, bus->domain_nr);
> + if (bridge->domain_nr == PCI_DOMAIN_NR_NOT_SET)
> + pci_bus_release_domain_nr(parent, bus->domain_nr);
> #endif
> if (bus_registered)
> put_device(&bus->dev);
> --
> 2.51.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms
2025-10-24 22:46 ` [PATCH v2 1/2] PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms Dan Williams
2025-10-25 19:53 ` Michael Kelley
@ 2025-10-27 20:08 ` Dave Jiang
1 sibling, 0 replies; 8+ messages in thread
From: Dave Jiang @ 2025-10-27 20:08 UTC (permalink / raw)
To: Dan Williams, bhelgaas
Cc: linux-pci, jonathan.derrick, lpieralisi, kwilczynski, mani, robh,
Suzuki K Poulose, Manivannan Sadhasivam, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Michael Kelley
On 10/24/25 3:46 PM, Dan Williams wrote:
> The ability to emulate a host bridge is useful not only for hardware PCI
> controllers like CONFIG_VMD, or virtual PCI controllers like
> CONFIG_PCI_HYPERV, but also for test and development scenarios like
> CONFIG_SAMPLES_DEVSEC [1].
>
> One stumbling block for defining CONFIG_SAMPLES_DEVSEC, a sample
> implementation of a platform TSM for PCI Device Security, is the need to
> accommodate PCI_DOMAINS_GENERIC architectures alongside x86 [2].
>
> In support of supplementing the existing CONFIG_PCI_BRIDGE_EMUL
> infrastructure for host bridges:
>
> * Introduce pci_bus_find_emul_domain_nr() as a common way to find a free
> PCI domain number whether that is to reuse the existing dynamic
> allocation code in the !ACPI case, or to assign an unused domain above
> the last ACPI segment.
>
> * Convert pci-hyperv to the new allocator so that the PCI core can
> unconditionally assume that bridge->domain_nr != PCI_DOMAIN_NR_NOT_SET
> is the dynamically allocated case.
>
> A follow on patch can also convert vmd to the new scheme. Currently vmd
> is limited to CONFIG_PCI_DOMAINS_GENERIC=n (x86) so, unlike pci-hyperv,
> it does not immediately conflict with this new
> pci_bus_find_emul_domain_nr() mechanism.
>
> Link: http://lore.kernel.org/174107249038.1288555.12362100502109498455.stgit@dwillia2-xfh.jf.intel.com [1]
> Reported-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> Closes: http://lore.kernel.org/20250311144601.145736-3-suzuki.poulose@arm.com
> Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>
> Cc: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: "K. Y. Srinivasan" <kys@microsoft.com>
> Cc: Haiyang Zhang <haiyangz@microsoft.com>
> Cc: Wei Liu <wei.liu@kernel.org>
> Cc: Dexuan Cui <decui@microsoft.com>
> Tested-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> [michael: maintain compatibility with userspace that expects 16-bit ids]
> Cc: Michael Kelley <mhklinux@outlook.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>> ---
> include/linux/pci.h | 7 ++++
> drivers/pci/controller/pci-hyperv.c | 62 +++++------------------------
> drivers/pci/pci.c | 24 ++++++++++-
> drivers/pci/probe.c | 8 +++-
> 4 files changed, 46 insertions(+), 55 deletions(-)
>
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index d1fdf81fbe1e..1ef1535802b0 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1956,10 +1956,17 @@ DEFINE_GUARD(pci_dev, struct pci_dev *, pci_dev_lock(_T), pci_dev_unlock(_T))
> */
> #ifdef CONFIG_PCI_DOMAINS
> extern int pci_domains_supported;
> +int pci_bus_find_emul_domain_nr(u32 hint, u32 min, u32 max);
> +void pci_bus_release_emul_domain_nr(int domain_nr);
> #else
> enum { pci_domains_supported = 0 };
> static inline int pci_domain_nr(struct pci_bus *bus) { return 0; }
> static inline int pci_proc_domain(struct pci_bus *bus) { return 0; }
> +static inline int pci_bus_find_emul_domain_nr(u32 hint, u32 min, u32 max)
> +{
> + return 0;
> +}
> +static inline void pci_bus_release_emul_domain_nr(int domain_nr) { }
> #endif /* CONFIG_PCI_DOMAINS */
>
> /*
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index 146b43981b27..64f68efaf547 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -3696,48 +3696,6 @@ static int hv_send_resources_released(struct hv_device *hdev)
> return 0;
> }
>
> -#define HVPCI_DOM_MAP_SIZE (64 * 1024)
> -static DECLARE_BITMAP(hvpci_dom_map, HVPCI_DOM_MAP_SIZE);
> -
> -/*
> - * PCI domain number 0 is used by emulated devices on Gen1 VMs, so define 0
> - * as invalid for passthrough PCI devices of this driver.
> - */
> -#define HVPCI_DOM_INVALID 0
> -
> -/**
> - * hv_get_dom_num() - Get a valid PCI domain number
> - * Check if the PCI domain number is in use, and return another number if
> - * it is in use.
> - *
> - * @dom: Requested domain number
> - *
> - * return: domain number on success, HVPCI_DOM_INVALID on failure
> - */
> -static u16 hv_get_dom_num(u16 dom)
> -{
> - unsigned int i;
> -
> - if (test_and_set_bit(dom, hvpci_dom_map) == 0)
> - return dom;
> -
> - for_each_clear_bit(i, hvpci_dom_map, HVPCI_DOM_MAP_SIZE) {
> - if (test_and_set_bit(i, hvpci_dom_map) == 0)
> - return i;
> - }
> -
> - return HVPCI_DOM_INVALID;
> -}
> -
> -/**
> - * hv_put_dom_num() - Mark the PCI domain number as free
> - * @dom: Domain number to be freed
> - */
> -static void hv_put_dom_num(u16 dom)
> -{
> - clear_bit(dom, hvpci_dom_map);
> -}
> -
> /**
> * hv_pci_probe() - New VMBus channel probe, for a root PCI bus
> * @hdev: VMBus's tracking struct for this root PCI bus
> @@ -3750,9 +3708,9 @@ static int hv_pci_probe(struct hv_device *hdev,
> {
> struct pci_host_bridge *bridge;
> struct hv_pcibus_device *hbus;
> - u16 dom_req, dom;
> + int ret, dom;
> + u16 dom_req;
> char *name;
> - int ret;
>
> bridge = devm_pci_alloc_host_bridge(&hdev->device, 0);
> if (!bridge)
> @@ -3779,11 +3737,14 @@ static int hv_pci_probe(struct hv_device *hdev,
> * PCI bus (which is actually emulated by the hypervisor) is domain 0.
> * (2) There will be no overlap between domains (after fixing possible
> * collisions) in the same VM.
> + *
> + * Because Gen1 VMs use domain 0, don't allow picking domain 0 here,
> + * even if bytes 4 and 5 of the instance GUID are both zero. For wider
> + * userspace compatibility, limit the domain id to a 16-bit value.
> */
> dom_req = hdev->dev_instance.b[5] << 8 | hdev->dev_instance.b[4];
> - dom = hv_get_dom_num(dom_req);
> -
> - if (dom == HVPCI_DOM_INVALID) {
> + dom = pci_bus_find_emul_domain_nr(dom_req, 1, U16_MAX);
> + if (dom < 0) {
> dev_err(&hdev->device,
> "Unable to use dom# 0x%x or other numbers", dom_req);
> ret = -EINVAL;
> @@ -3917,7 +3878,7 @@ static int hv_pci_probe(struct hv_device *hdev,
> destroy_wq:
> destroy_workqueue(hbus->wq);
> free_dom:
> - hv_put_dom_num(hbus->bridge->domain_nr);
> + pci_bus_release_emul_domain_nr(hbus->bridge->domain_nr);
> free_bus:
> kfree(hbus);
> return ret;
> @@ -4042,8 +4003,6 @@ static void hv_pci_remove(struct hv_device *hdev)
> irq_domain_remove(hbus->irq_domain);
> irq_domain_free_fwnode(hbus->fwnode);
>
> - hv_put_dom_num(hbus->bridge->domain_nr);
> -
> kfree(hbus);
> }
>
> @@ -4217,9 +4176,6 @@ static int __init init_hv_pci_drv(void)
> if (ret)
> return ret;
>
> - /* Set the invalid domain number's bit, so it will not be used */
> - set_bit(HVPCI_DOM_INVALID, hvpci_dom_map);
> -
> /* Initialize PCI block r/w interface */
> hvpci_block_ops.read_block = hv_read_config_block;
> hvpci_block_ops.write_block = hv_write_config_block;
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index b14dd064006c..64aed8705e91 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -6656,9 +6656,31 @@ static void pci_no_domains(void)
> #endif
> }
>
> +#ifdef CONFIG_PCI_DOMAINS
> +static DEFINE_IDA(pci_domain_nr_dynamic_ida);
> +
> +/**
> + * pci_bus_find_emul_domain_nr() - allocate a PCI domain number per constraints
> + * @hint: desired domain, 0 if any id in the range of @min to @max is acceptable
> + * @min: minimum allowable domain
> + * @max: maximum allowable domain, no ids higher than INT_MAX will be returned
> + */
> +int pci_bus_find_emul_domain_nr(u32 hint, u32 min, u32 max)
> +{
> + return ida_alloc_range(&pci_domain_nr_dynamic_ida, max(hint, min), max,
> + GFP_KERNEL);
> +}
> +EXPORT_SYMBOL_GPL(pci_bus_find_emul_domain_nr);
> +
> +void pci_bus_release_emul_domain_nr(int domain_nr)
> +{
> + ida_free(&pci_domain_nr_dynamic_ida, domain_nr);
> +}
> +EXPORT_SYMBOL_GPL(pci_bus_release_emul_domain_nr);
> +#endif
> +
> #ifdef CONFIG_PCI_DOMAINS_GENERIC
> static DEFINE_IDA(pci_domain_nr_static_ida);
> -static DEFINE_IDA(pci_domain_nr_dynamic_ida);
>
> static void of_pci_reserve_static_domain_nr(void)
> {
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 0ce98e18b5a8..5e101ced00a5 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -650,6 +650,11 @@ static void pci_release_host_bridge_dev(struct device *dev)
>
> pci_free_resource_list(&bridge->windows);
> pci_free_resource_list(&bridge->dma_ranges);
> +
> + /* Host bridges only have domain_nr set in the emulation case */
> + if (bridge->domain_nr != PCI_DOMAIN_NR_NOT_SET)
> + pci_bus_release_emul_domain_nr(bridge->domain_nr);
> +
> kfree(bridge);
> }
>
> @@ -1130,7 +1135,8 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
> device_del(&bridge->dev);
> free:
> #ifdef CONFIG_PCI_DOMAINS_GENERIC
> - pci_bus_release_domain_nr(parent, bus->domain_nr);
> + if (bridge->domain_nr == PCI_DOMAIN_NR_NOT_SET)
> + pci_bus_release_domain_nr(parent, bus->domain_nr);
> #endif
> if (bus_registered)
> put_device(&bus->dev);
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 2/2] PCI: vmd: Switch to pci_bus_find_emul_domain_nr()
2025-10-24 22:46 ` [PATCH v2 2/2] PCI: vmd: Switch to pci_bus_find_emul_domain_nr() Dan Williams
@ 2025-10-27 20:08 ` Dave Jiang
0 siblings, 0 replies; 8+ messages in thread
From: Dave Jiang @ 2025-10-27 20:08 UTC (permalink / raw)
To: Dan Williams, bhelgaas
Cc: linux-pci, jonathan.derrick, lpieralisi, kwilczynski, mani, robh,
Szymon Durawa, Nirmal Patel
On 10/24/25 3:46 PM, Dan Williams wrote:
> The new common domain number allocator can replace the custom allocator
> in VMD.
>
> Beyond some code reuse benefits it does close a potential race whereby
> vmd_find_free_domain() collides with new PCI buses coming online with a
> conflicting domain number. Such a race has not been observed in
> practice, hence not tagging this change as a fix.
>
> As VMD uses pci_create_root_bus() rather than pci_alloc_host_bridge() +
> pci_scan_root_bus_bridge() it has no chance to set ->domain_nr in the
> bridge so needs to manage freeing the domain number on its own.
>
> Cc: Szymon Durawa <szymon.durawa@linux.intel.com>
> Cc: Nirmal Patel <nirmal.patel@linux.intel.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>> ---
> drivers/pci/controller/vmd.c | 40 +++++++++++++++---------------------
> 1 file changed, 17 insertions(+), 23 deletions(-)
>
> diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> index b4b62b9ccc45..03b5920138f0 100644
> --- a/drivers/pci/controller/vmd.c
> +++ b/drivers/pci/controller/vmd.c
> @@ -578,22 +578,6 @@ static void vmd_detach_resources(struct vmd_dev *vmd)
> vmd->dev->resource[VMD_MEMBAR2].child = NULL;
> }
>
> -/*
> - * VMD domains start at 0x10000 to not clash with ACPI _SEG domains.
> - * Per ACPI r6.0, sec 6.5.6, _SEG returns an integer, of which the lower
> - * 16 bits are the PCI Segment Group (domain) number. Other bits are
> - * currently reserved.
> - */
> -static int vmd_find_free_domain(void)
> -{
> - int domain = 0xffff;
> - struct pci_bus *bus = NULL;
> -
> - while ((bus = pci_find_next_bus(bus)) != NULL)
> - domain = max_t(int, domain, pci_domain_nr(bus));
> - return domain + 1;
> -}
> -
> static int vmd_get_phys_offsets(struct vmd_dev *vmd, bool native_hint,
> resource_size_t *offset1,
> resource_size_t *offset2)
> @@ -878,13 +862,6 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
> .parent = res,
> };
>
> - sd->vmd_dev = vmd->dev;
> - sd->domain = vmd_find_free_domain();
> - if (sd->domain < 0)
> - return sd->domain;
> -
> - sd->node = pcibus_to_node(vmd->dev->bus);
> -
> /*
> * Currently MSI remapping must be enabled in guest passthrough mode
> * due to some missing interrupt remapping plumbing. This is probably
> @@ -910,9 +887,24 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
> pci_add_resource_offset(&resources, &vmd->resources[1], offset[0]);
> pci_add_resource_offset(&resources, &vmd->resources[2], offset[1]);
>
> + sd->vmd_dev = vmd->dev;
> +
> + /*
> + * Emulated domains start at 0x10000 to not clash with ACPI _SEG
> + * domains. Per ACPI r6.0, sec 6.5.6, _SEG returns an integer, of
> + * which the lower 16 bits are the PCI Segment Group (domain) number.
> + * Other bits are currently reserved.
> + */
> + sd->domain = pci_bus_find_emul_domain_nr(0, 0x10000, INT_MAX);
> + if (sd->domain < 0)
> + return sd->domain;
> +
> + sd->node = pcibus_to_node(vmd->dev->bus);
> +
> vmd->bus = pci_create_root_bus(&vmd->dev->dev, vmd->busn_start,
> &vmd_ops, sd, &resources);
> if (!vmd->bus) {
> + pci_bus_release_emul_domain_nr(sd->domain);
> pci_free_resource_list(&resources);
> vmd_remove_irq_domain(vmd);
> return -ENODEV;
> @@ -1005,6 +997,7 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
> return -ENOMEM;
>
> vmd->dev = dev;
> + vmd->sysdata.domain = PCI_DOMAIN_NR_NOT_SET;
> vmd->instance = ida_alloc(&vmd_instance_ida, GFP_KERNEL);
> if (vmd->instance < 0)
> return vmd->instance;
> @@ -1070,6 +1063,7 @@ static void vmd_remove(struct pci_dev *dev)
> vmd_detach_resources(vmd);
> vmd_remove_irq_domain(vmd);
> ida_free(&vmd_instance_ida, vmd->instance);
> + pci_bus_release_emul_domain_nr(vmd->sysdata.domain);
> }
>
> static void vmd_shutdown(struct pci_dev *dev)
^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH v2 1/2] PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms
2025-10-25 19:53 ` Michael Kelley
@ 2025-10-27 23:45 ` dan.j.williams
0 siblings, 0 replies; 8+ messages in thread
From: dan.j.williams @ 2025-10-27 23:45 UTC (permalink / raw)
To: Michael Kelley, Dan Williams, bhelgaas@google.com
Cc: linux-pci@vger.kernel.org, jonathan.derrick@linux.dev,
lpieralisi@kernel.org, kwilczynski@kernel.org, mani@kernel.org,
robh@kernel.org, Suzuki K Poulose, Manivannan Sadhasivam,
K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui
Michael Kelley wrote:
> From: Dan Williams <dan.j.williams@intel.com> Sent: Friday, October 24, 2025 3:46 PM
> >
> > The ability to emulate a host bridge is useful not only for hardware PCI
> > controllers like CONFIG_VMD, or virtual PCI controllers like
> > CONFIG_PCI_HYPERV, but also for test and development scenarios like
> > CONFIG_SAMPLES_DEVSEC [1].
> >
> > One stumbling block for defining CONFIG_SAMPLES_DEVSEC, a sample
> > implementation of a platform TSM for PCI Device Security, is the need to
> > accommodate PCI_DOMAINS_GENERIC architectures alongside x86 [2].
>
> There's not a [2] tag anywhere below. Presumably it should be the "Closes:"
> link?
Yes, good catch.
> > In support of supplementing the existing CONFIG_PCI_BRIDGE_EMUL
> > infrastructure for host bridges:
> >
> > * Introduce pci_bus_find_emul_domain_nr() as a common way to find a free
> > PCI domain number whether that is to reuse the existing dynamic
> > allocation code in the !ACPI case, or to assign an unused domain above
> > the last ACPI segment.
> >
> > * Convert pci-hyperv to the new allocator so that the PCI core can
> > unconditionally assume that bridge->domain_nr != PCI_DOMAIN_NR_NOT_SET
> > is the dynamically allocated case.
> >
> > A follow on patch can also convert vmd to the new scheme. Currently vmd
> > is limited to CONFIG_PCI_DOMAINS_GENERIC=n (x86) so, unlike pci-hyperv,
> > it does not immediately conflict with this new
> > pci_bus_find_emul_domain_nr() mechanism.
> >
> > Link: https://lore.kernel.org/all/174107249038.1288555.12362100502109498455.stgit@dwillia2-xfh.jf.intel.com [1]
> > Reported-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> > Closes: https://lore.kernel.org/all/20250311144601.145736-3-suzuki.poulose@arm.com
> > Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>
> > Cc: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > Cc: Rob Herring <robh@kernel.org>
> > Cc: Bjorn Helgaas <bhelgaas@google.com>
> > Cc: "K. Y. Srinivasan" <kys@microsoft.com>
> > Cc: Haiyang Zhang <haiyangz@microsoft.com>
> > Cc: Wei Liu <wei.liu@kernel.org>
> > Cc: Dexuan Cui <decui@microsoft.com>
> > Tested-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> > [michael: maintain compatibility with userspace that expects 16-bit ids]
>
> Is the above line spurious? It doesn't seem to belong here.
That was documenting the change to make the @max argument to
pci_bus_find_emul_domain_nr() be U16_MAX in the Hyper-V case, but yeah
no need to note that in this case. You did not edit the patch directly.
I am ok if Bjorn deletes that on applying.
If you want a respin Bjorn, let me know.
> > Cc: Michael Kelley <mhklinux@outlook.com>
> > Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>
> Tested on x86 and arm64 VMs in the Azure public cloud with
> multiple Hyper-V virtual PCI devices in each VM. So covered
> both the "CONFIG_PCI_DOMAINS_GENERIC=n" and "=y" cases.
> Did manual unbind/rebind of vPCI devices multiple times so the
> domain numbers would be freed and reallocated. All was good.
> I did not go to the trouble of simulating a domain number
> collision.
>
> Reviewed-by: Michael Kelley <mhklinux@outlook.com>
> Tested-by: Michael Kelley <mhklinux@outlook.com>
Thanks, Michael much appreciated!
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 0/2] PCI: Unify domain emulation
2025-10-24 22:46 [PATCH v2 0/2] PCI: Unify domain emulation Dan Williams
2025-10-24 22:46 ` [PATCH v2 1/2] PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms Dan Williams
2025-10-24 22:46 ` [PATCH v2 2/2] PCI: vmd: Switch to pci_bus_find_emul_domain_nr() Dan Williams
@ 2025-10-28 17:39 ` Bjorn Helgaas
2 siblings, 0 replies; 8+ messages in thread
From: Bjorn Helgaas @ 2025-10-28 17:39 UTC (permalink / raw)
To: Dan Williams
Cc: bhelgaas, linux-pci, jonathan.derrick, lpieralisi, kwilczynski,
mani, robh, Dexuan Cui, Haiyang Zhang, K. Y. Srinivasan,
Manivannan Sadhasivam, Michael Kelley, Nirmal Patel,
Suzuki K Poulose, Szymon Durawa, Wei Liu
On Fri, Oct 24, 2025 at 03:46:20PM -0700, Dan Williams wrote:
> Changes since v1 [1]:
> - Rebase on v6.18-rc2
> - Support callers supplying both a hint and a range for the Hyper-V hint
> + fallback case (Michael)
> - Add comment explaining domain number 0 vs Gen1 VMs, and domain values
> greater than U16_MAX concerns (Michael)
> - Leave the VMD status quo comment about requesting domain numbers >
> U16_MAX
>
> [1]: http://lore.kernel.org/20250716160835.680486-1-dan.j.williams@intel.com
>
> The PCI/TSM effort created a sample driver to test ABI flows
> (samples/devsec/ [2]). Suzuki observed that it only worked on x86 due to
> its dependency on CONFIG_PCI_DOMAINS_GENERIC=n. I.e. an unfortunate
> restriction for what should be an architecture agnostic test framework.
>
> Introduce a new pci_bus_find_emul_domain_nr() helper that all "soft"
> host-bridge drivers can share and hide the CONFIG_PCI_DOMAINS_GENERIC
> details behind that helper.
>
> [2]: https://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm.git/commit/?id=0e16ce0b9c64
>
> Dan Williams (2):
> PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms
> PCI: vmd: Switch to pci_bus_find_emul_domain_nr()
>
> include/linux/pci.h | 7 ++++
> drivers/pci/controller/pci-hyperv.c | 62 +++++------------------------
> drivers/pci/controller/vmd.c | 40 ++++++++-----------
> drivers/pci/pci.c | 24 ++++++++++-
> drivers/pci/probe.c | 8 +++-
> 5 files changed, 63 insertions(+), 78 deletions(-)
Applied to pci/enumeration for v6.19, thanks, Dan!
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-10-28 17:39 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-24 22:46 [PATCH v2 0/2] PCI: Unify domain emulation Dan Williams
2025-10-24 22:46 ` [PATCH v2 1/2] PCI: Enable host bridge emulation for PCI_DOMAINS_GENERIC platforms Dan Williams
2025-10-25 19:53 ` Michael Kelley
2025-10-27 23:45 ` dan.j.williams
2025-10-27 20:08 ` Dave Jiang
2025-10-24 22:46 ` [PATCH v2 2/2] PCI: vmd: Switch to pci_bus_find_emul_domain_nr() Dan Williams
2025-10-27 20:08 ` Dave Jiang
2025-10-28 17:39 ` [PATCH v2 0/2] PCI: Unify domain emulation Bjorn Helgaas
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).