[PATCH v6 0/9] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[PATCH v6 0/9] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Jiang Liu]

From: Jiang Liu <liuj97@gmail.com>

From: Jiang Liu <jiang.liu@huawei.com>

This patchset enhance pci_root driver to update MMCFG information when
hot-plugging PCI root bridges. It applies to 
git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git pci/next-3.5

-v2: split into smaller patches and skip updating MMCFG information when
     MMCFG is disabled
-v3: add mmconf_added to simply free path, also make pci_mmconfig_insert()
     to process extra exist case --- By Yinghai
-v4: tune arch_acpi_pci_root_add() to handle a corner case raised by Kenji
-v5: address review comments from Bjorn and Taku, also better handle corner
     cases in arch_acpi_pci_root_add()
-v6: get rid of arch_acpi_pci_root_xxx() by using existing hooks
     add MCFG information for host bridges on demand
     more corner cases clear up
     correctly handle condition compilation
     fix section mismatch issues
     fix a issue reported by Taku about a BIOS bug

The first 4 patches in series is the same with v5.

On a test platform with four SandyBridge processors, MMCFG resources from
/proc/iomem are as below.
With original implementation:
bf000000-cfffffff : reserved
  c0000000-cfffffff : PCI MMCONFIG 0000 [bus 00-ff]

With patch set applied:
bf000000-cfffffff : reserved
  c0000000-c3efffff : PCI MMCONFIG 0000 [bus 00-3e]
  c3f00000-c3ffffff : PCI MMCONFIG 0000 [bus 3f-3f]
  c4000000-c7efffff : PCI MMCONFIG 0000 [bus 40-7e]
  c7f00000-c7ffffff : PCI MMCONFIG 0000 [bus 7f-7f]
  c8000000-cbefffff : PCI MMCONFIG 0000 [bus 80-be]
  cbf00000-cbffffff : PCI MMCONFIG 0000 [bus bf-bf]
  cc000000-cfefffff : PCI MMCONFIG 0000 [bus c0-fe]
  cff00000-cfffffff : PCI MMCONFIG 0000 [bus ff-ff]

After removing a host bridge:
bf000000-cfffffff : reserved
  c0000000-c3efffff : PCI MMCONFIG 0000 [bus 00-3e]
  c3f00000-c3ffffff : PCI MMCONFIG 0000 [bus 3f-3f]
  c4000000-c7efffff : PCI MMCONFIG 0000 [bus 40-7e]
  c7f00000-c7ffffff : PCI MMCONFIG 0000 [bus 7f-7f]
  c8000000-cbefffff : PCI MMCONFIG 0000 [bus 80-be]
  cbf00000-cbffffff : PCI MMCONFIG 0000 [bus bf-bf]
  cc000000-cfefffff : PCI MMCONFIG 0000 [bus c0-fe]
  cff00000-cfffffff : PCI MMCONFIG 0000 [bus ff-ff]

After adding the host bridge back
bf000000-cfffffff : reserved
  c0000000-c3efffff : PCI MMCONFIG 0000 [bus 00-3e]
  c3f00000-c3ffffff : PCI MMCONFIG 0000 [bus 3f-3f]
  c4000000-c7efffff : PCI MMCONFIG 0000 [bus 40-7e]
  c7f00000-c7ffffff : PCI MMCONFIG 0000 [bus 7f-7f]
  c8000000-cbefffff : PCI MMCONFIG 0000 [bus 80-be]
  cbf00000-cbffffff : PCI MMCONFIG 0000 [bus bf-bf]
  cc000000-cfefffff : PCI MMCONFIG 0000 [bus c0-fe]
  cff00000-cfffffff : PCI MMCONFIG 0000 [bus ff-ff]

Jiang Liu (9):
  PCI, x86: split out pci_mmcfg_check_reserved() for code reuse
  PCI, x86: split out pci_mmconfig_alloc() for code reuse
  PCI, x86: use RCU list to protect mmconfig list
  PCI, x86: introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap()
  PCI, x86: introduce pci_mmconfig_insert()/delete() for PCI root
    bridge hotplug
  PCI, ACPI: provide MCFG address for PCI host bridges
  PCI, x86: update MMCFG information when hot-plugging PCI host bridges
  PCI, x86: add MMCFG information on demand
  PCI, x86: simplify pci_mmcfg_late_insert_resources()

 arch/x86/include/asm/pci_x86.h |   11 ++
 arch/x86/pci/acpi.c            |   71 ++++++++++
 arch/x86/pci/legacy.c          |    1 +
 arch/x86/pci/mmconfig-shared.c |  280 ++++++++++++++++++++++++++++++----------
 arch/x86/pci/mmconfig_32.c     |   30 ++++-
 arch/x86/pci/mmconfig_64.c     |   37 +++++-
 drivers/acpi/pci_root.c        |   12 ++
 drivers/pci/pci-acpi.c         |   34 +++++
 include/acpi/acnames.h         |    1 +
 include/acpi/acpi_bus.h        |    3 +
 include/linux/pci-acpi.h       |    5 +
 11 files changed, 412 insertions(+), 73 deletions(-)

[PATCH v6 1/9] PCI, x86: split out pci_mmcfg_check_reserved() for code reuse

[Jiang Liu]

From: Jiang Liu <liuj97@gmail.com>

From: Jiang Liu <jiang.liu@huawei.com>

Split out pci_mmcfg_check_reserved() for code reuse, which will be used
when supporting PCI host bridge hotplug.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/pci/mmconfig-shared.c |   51 +++++++++++++++++++--------------------
 1 files changed, 25 insertions(+), 26 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 301e325..f799949 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -474,39 +474,38 @@ static int __init is_mmconf_reserved(check_reserved_t is_reserved,
 	return valid;
 }
 
+static int __devinit pci_mmcfg_check_reserved(struct pci_mmcfg_region *cfg,
+					      int early)
+{
+	if (!early && !acpi_disabled) {
+		if (is_mmconf_reserved(is_acpi_reserved, cfg, 0))
+			return 1;
+		else
+			printk(KERN_ERR FW_BUG PREFIX
+			       "MMCONFIG at %pR not reserved in "
+			       "ACPI motherboard resources\n",
+			       &cfg->res);
+	}
+
+	/* Don't try to do this check unless configuration
+	   type 1 is available. how about type 2 ?*/
+	if (raw_pci_ops)
+		return is_mmconf_reserved(e820_all_mapped, cfg, 1);
+
+	return 0;
+}
+
 static void __init pci_mmcfg_reject_broken(int early)
 {
 	struct pci_mmcfg_region *cfg;
 
 	list_for_each_entry(cfg, &pci_mmcfg_list, list) {
-		int valid = 0;
-
-		if (!early && !acpi_disabled) {
-			valid = is_mmconf_reserved(is_acpi_reserved, cfg, 0);
-
-			if (valid)
-				continue;
-			else
-				printk(KERN_ERR FW_BUG PREFIX
-				       "MMCONFIG at %pR not reserved in "
-				       "ACPI motherboard resources\n",
-				       &cfg->res);
+		if (pci_mmcfg_check_reserved(cfg, early) == 0) {
+			printk(KERN_INFO PREFIX "not using MMCONFIG\n");
+			free_all_mmcfg();
+			return;
 		}
-
-		/* Don't try to do this check unless configuration
-		   type 1 is available. how about type 2 ?*/
-		if (raw_pci_ops)
-			valid = is_mmconf_reserved(e820_all_mapped, cfg, 1);
-
-		if (!valid)
-			goto reject;
 	}
-
-	return;
-
-reject:
-	printk(KERN_INFO PREFIX "not using MMCONFIG\n");
-	free_all_mmcfg();
 }
 
 static int __initdata known_bridge;
-- 
1.7.1

[PATCH v6 2/9] PCI, x86: split out pci_mmconfig_alloc() for code reuse

[Jiang Liu]

From: Jiang Liu <liuj97@gmail.com>

From: Jiang Liu <jiang.liu@huawei.com>

Split out pci_mmconfig_alloc() for code reuse, which will be used
when supporting PCI root bridge hotplug.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/pci/mmconfig-shared.c |   19 +++++++++++++++----
 1 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index f799949..5e2cd2a 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -61,8 +61,9 @@ static __init void list_add_sorted(struct pci_mmcfg_region *new)
 	list_add_tail(&new->list, &pci_mmcfg_list);
 }
 
-static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
-							int end, u64 addr)
+static __devinit struct pci_mmcfg_region *pci_mmconfig_alloc(int segment,
+							     int start,
+							     int end, u64 addr)
 {
 	struct pci_mmcfg_region *new;
 	struct resource *res;
@@ -79,8 +80,6 @@ static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
 	new->start_bus = start;
 	new->end_bus = end;
 
-	list_add_sorted(new);
-
 	res = &new->res;
 	res->start = addr + PCI_MMCFG_BUS_OFFSET(start);
 	res->end = addr + PCI_MMCFG_BUS_OFFSET(end + 1) - 1;
@@ -96,6 +95,18 @@ static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
 	return new;
 }
 
+static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
+							int end, u64 addr)
+{
+	struct pci_mmcfg_region *new;
+
+	new = pci_mmconfig_alloc(segment, start, end, addr);
+	if (new)
+		list_add_sorted(new);
+
+	return new;
+}
+
 struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus)
 {
 	struct pci_mmcfg_region *cfg;
-- 
1.7.1

[PATCH v6 3/9] PCI, x86: use RCU list to protect mmconfig list

[Jiang Liu]

From: Jiang Liu <liuj97@gmail.com>

From: Jiang Liu <jiang.liu@huawei.com>

Use RCU list to protect mmconfig list from dynamic change
when supporting PCI host bridge hotplug.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/pci/mmconfig-shared.c |   18 ++++++++++++------
 arch/x86/pci/mmconfig_32.c     |   13 +++++++++++--
 arch/x86/pci/mmconfig_64.c     |   13 +++++++++++--
 3 files changed, 34 insertions(+), 10 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 5e2cd2a..0ac97d5 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -17,6 +17,8 @@
 #include <linux/bitmap.h>
 #include <linux/dmi.h>
 #include <linux/slab.h>
+#include <linux/mutex.h>
+#include <linux/rculist.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 #include <asm/acpi.h>
@@ -25,6 +27,7 @@
 
 /* Indicate if the mmcfg resources have been placed into the resource table. */
 static int __initdata pci_mmcfg_resources_inserted;
+static DEFINE_MUTEX(pci_mmcfg_lock);
 
 LIST_HEAD(pci_mmcfg_list);
 
@@ -45,20 +48,20 @@ static __init void free_all_mmcfg(void)
 		pci_mmconfig_remove(cfg);
 }
 
-static __init void list_add_sorted(struct pci_mmcfg_region *new)
+static __devinit void list_add_sorted(struct pci_mmcfg_region *new)
 {
 	struct pci_mmcfg_region *cfg;
 
 	/* keep list sorted by segment and starting bus number */
-	list_for_each_entry(cfg, &pci_mmcfg_list, list) {
+	list_for_each_entry_rcu(cfg, &pci_mmcfg_list, list) {
 		if (cfg->segment > new->segment ||
 		    (cfg->segment == new->segment &&
 		     cfg->start_bus >= new->start_bus)) {
-			list_add_tail(&new->list, &cfg->list);
+			list_add_tail_rcu(&new->list, &cfg->list);
 			return;
 		}
 	}
-	list_add_tail(&new->list, &pci_mmcfg_list);
+	list_add_tail_rcu(&new->list, &pci_mmcfg_list);
 }
 
 static __devinit struct pci_mmcfg_region *pci_mmconfig_alloc(int segment,
@@ -101,8 +104,11 @@ static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
 	struct pci_mmcfg_region *new;
 
 	new = pci_mmconfig_alloc(segment, start, end, addr);
-	if (new)
+	if (new) {
+		mutex_lock(&pci_mmcfg_lock);
 		list_add_sorted(new);
+		mutex_unlock(&pci_mmcfg_lock);
+	}
 
 	return new;
 }
@@ -111,7 +117,7 @@ struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus)
 {
 	struct pci_mmcfg_region *cfg;
 
-	list_for_each_entry(cfg, &pci_mmcfg_list, list)
+	list_for_each_entry_rcu(cfg, &pci_mmcfg_list, list)
 		if (cfg->segment == segment &&
 		    cfg->start_bus <= bus && bus <= cfg->end_bus)
 			return cfg;
diff --git a/arch/x86/pci/mmconfig_32.c b/arch/x86/pci/mmconfig_32.c
index 5372e86..5dad04a 100644
--- a/arch/x86/pci/mmconfig_32.c
+++ b/arch/x86/pci/mmconfig_32.c
@@ -11,6 +11,7 @@
 
 #include <linux/pci.h>
 #include <linux/init.h>
+#include <linux/rcupdate.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 #include <acpi/acpi.h>
@@ -60,9 +61,12 @@ err:		*value = -1;
 		return -EINVAL;
 	}
 
+	rcu_read_lock();
 	base = get_base_addr(seg, bus, devfn);
-	if (!base)
+	if (!base) {
+		rcu_read_unlock();
 		goto err;
+	}
 
 	raw_spin_lock_irqsave(&pci_config_lock, flags);
 
@@ -80,6 +84,7 @@ err:		*value = -1;
 		break;
 	}
 	raw_spin_unlock_irqrestore(&pci_config_lock, flags);
+	rcu_read_unlock();
 
 	return 0;
 }
@@ -93,9 +98,12 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 	if ((bus > 255) || (devfn > 255) || (reg > 4095))
 		return -EINVAL;
 
+	rcu_read_lock();
 	base = get_base_addr(seg, bus, devfn);
-	if (!base)
+	if (!base) {
+		rcu_read_unlock();
 		return -EINVAL;
+	}
 
 	raw_spin_lock_irqsave(&pci_config_lock, flags);
 
@@ -113,6 +121,7 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 		break;
 	}
 	raw_spin_unlock_irqrestore(&pci_config_lock, flags);
+	rcu_read_unlock();
 
 	return 0;
 }
diff --git a/arch/x86/pci/mmconfig_64.c b/arch/x86/pci/mmconfig_64.c
index 915a493..acc48c5 100644
--- a/arch/x86/pci/mmconfig_64.c
+++ b/arch/x86/pci/mmconfig_64.c
@@ -9,6 +9,7 @@
 #include <linux/init.h>
 #include <linux/acpi.h>
 #include <linux/bitmap.h>
+#include <linux/rcupdate.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 
@@ -34,9 +35,12 @@ err:		*value = -1;
 		return -EINVAL;
 	}
 
+	rcu_read_lock();
 	addr = pci_dev_base(seg, bus, devfn);
-	if (!addr)
+	if (!addr) {
+		rcu_read_unlock();
 		goto err;
+	}
 
 	switch (len) {
 	case 1:
@@ -49,6 +53,7 @@ err:		*value = -1;
 		*value = mmio_config_readl(addr + reg);
 		break;
 	}
+	rcu_read_unlock();
 
 	return 0;
 }
@@ -62,9 +67,12 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 	if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095)))
 		return -EINVAL;
 
+	rcu_read_lock();
 	addr = pci_dev_base(seg, bus, devfn);
-	if (!addr)
+	if (!addr) {
+		rcu_read_unlock();
 		return -EINVAL;
+	}
 
 	switch (len) {
 	case 1:
@@ -77,6 +85,7 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 		mmio_config_writel(addr + reg, value);
 		break;
 	}
+	rcu_read_unlock();
 
 	return 0;
 }
-- 
1.7.1

[PATCH v6 4/9] PCI, x86: introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap()

[Jiang Liu]

From: Jiang Liu <liuj97@gmail.com>

From: Jiang Liu <jiang.liu@huawei.com>

Introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap(), which will be used
when supporting PCI root bridge hotplug.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/include/asm/pci_x86.h |    2 ++
 arch/x86/pci/mmconfig_32.c     |   15 +++++++++++++++
 arch/x86/pci/mmconfig_64.c     |   22 +++++++++++++++++++++-
 3 files changed, 38 insertions(+), 1 deletions(-)

diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index b3a5317..df898ce 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -135,6 +135,8 @@ struct pci_mmcfg_region {
 
 extern int __init pci_mmcfg_arch_init(void);
 extern void __init pci_mmcfg_arch_free(void);
+extern int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg);
+extern void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg);
 extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus);
 
 extern struct list_head pci_mmcfg_list;
diff --git a/arch/x86/pci/mmconfig_32.c b/arch/x86/pci/mmconfig_32.c
index 5dad04a..a22785d 100644
--- a/arch/x86/pci/mmconfig_32.c
+++ b/arch/x86/pci/mmconfig_32.c
@@ -141,3 +141,18 @@ int __init pci_mmcfg_arch_init(void)
 void __init pci_mmcfg_arch_free(void)
 {
 }
+
+int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg)
+{
+	return 0;
+}
+
+void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg)
+{
+	unsigned long flags;
+
+	/* Invalidate the cached mmcfg map entry. */
+	raw_spin_lock_irqsave(&pci_config_lock, flags);
+	mmcfg_last_accessed_device = 0;
+	raw_spin_unlock_irqrestore(&pci_config_lock, flags);
+}
diff --git a/arch/x86/pci/mmconfig_64.c b/arch/x86/pci/mmconfig_64.c
index acc48c5..4e05779 100644
--- a/arch/x86/pci/mmconfig_64.c
+++ b/arch/x86/pci/mmconfig_64.c
@@ -95,7 +95,7 @@ static const struct pci_raw_ops pci_mmcfg = {
 	.write =	pci_mmcfg_write,
 };
 
-static void __iomem * __init mcfg_ioremap(struct pci_mmcfg_region *cfg)
+static void __iomem * __devinit mcfg_ioremap(struct pci_mmcfg_region *cfg)
 {
 	void __iomem *addr;
 	u64 start, size;
@@ -138,3 +138,23 @@ void __init pci_mmcfg_arch_free(void)
 		}
 	}
 }
+
+int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg)
+{
+	cfg->virt = mcfg_ioremap(cfg);
+	if (!cfg->virt) {
+		printk(KERN_ERR PREFIX "can't map MMCONFIG at %pR\n",
+		       &cfg->res);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg)
+{
+	if (cfg && cfg->virt) {
+		iounmap(cfg->virt + PCI_MMCFG_BUS_OFFSET(cfg->start_bus));
+		cfg->virt = NULL;
+	}
+}
-- 
1.7.1

[PATCH v6 5/9] PCI, x86: introduce pci_mmconfig_insert()/delete() for PCI root bridge hotplug

[Jiang Liu]

From: Jiang Liu <liuj97@gmail.com>

From: Jiang Liu <jiang.liu@huawei.com>

Introduce pci_mmconfig_insert()/pci_mmconfig_delete(), which will be used to
update MMCFG information when supporting PCI root bridge hotplug.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/include/asm/pci_x86.h |    3 +
 arch/x86/pci/mmconfig-shared.c |  130 +++++++++++++++++++++++++++++++++++++---
 2 files changed, 124 insertions(+), 9 deletions(-)

diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index df898ce..1a3c12f 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -137,6 +137,9 @@ extern int __init pci_mmcfg_arch_init(void);
 extern void __init pci_mmcfg_arch_free(void);
 extern int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg);
 extern void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg);
+extern int __devinit pci_mmconfig_insert(uint16_t seg, uint8_t start,
+					 uint8_t end, phys_addr_t addr);
+extern int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end);
 extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus);
 
 extern struct list_head pci_mmcfg_list;
diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 0ac97d5..8d0c287 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -27,6 +27,7 @@
 
 /* Indicate if the mmcfg resources have been placed into the resource table. */
 static int __initdata pci_mmcfg_resources_inserted;
+static bool pci_mmcfg_arch_init_failed;
 static DEFINE_MUTEX(pci_mmcfg_lock);
 
 LIST_HEAD(pci_mmcfg_list);
@@ -374,15 +375,20 @@ static void __init pci_mmcfg_insert_resources(void)
 {
 	struct pci_mmcfg_region *cfg;
 
+	/*
+	 * Insert resources for MMCFG items if the resource hasn't been
+	 * inserted by pci_mmconfig_insert() yet.
+	 */
 	list_for_each_entry(cfg, &pci_mmcfg_list, list)
-		insert_resource(&iomem_resource, &cfg->res);
+		if (!cfg->res.parent)
+			insert_resource(&iomem_resource, &cfg->res);
 
 	/* Mark that the resources have been inserted. */
 	pci_mmcfg_resources_inserted = 1;
 }
 
-static acpi_status __init check_mcfg_resource(struct acpi_resource *res,
-					      void *data)
+static acpi_status __devinit check_mcfg_resource(struct acpi_resource *res,
+						 void *data)
 {
 	struct resource *mcfg_res = data;
 	struct acpi_resource_address64 address;
@@ -418,8 +424,8 @@ static acpi_status __init check_mcfg_resource(struct acpi_resource *res,
 	return AE_OK;
 }
 
-static acpi_status __init find_mboard_resource(acpi_handle handle, u32 lvl,
-		void *context, void **rv)
+static acpi_status __devinit find_mboard_resource(acpi_handle handle, u32 lvl,
+						  void *context, void **rv)
 {
 	struct resource *mcfg_res = context;
 
@@ -432,7 +438,7 @@ static acpi_status __init find_mboard_resource(acpi_handle handle, u32 lvl,
 	return AE_OK;
 }
 
-static int __init is_acpi_reserved(u64 start, u64 end, unsigned not_used)
+static int __devinit is_acpi_reserved(u64 start, u64 end, unsigned not_used)
 {
 	struct resource mcfg_res;
 
@@ -451,8 +457,9 @@ static int __init is_acpi_reserved(u64 start, u64 end, unsigned not_used)
 
 typedef int (*check_reserved_t)(u64 start, u64 end, unsigned type);
 
-static int __init is_mmconf_reserved(check_reserved_t is_reserved,
-				    struct pci_mmcfg_region *cfg, int with_e820)
+static int __devinit is_mmconf_reserved(check_reserved_t is_reserved,
+					struct pci_mmcfg_region *cfg,
+					int with_e820)
 {
 	u64 addr = cfg->res.start;
 	u64 size = resource_size(&cfg->res);
@@ -491,7 +498,7 @@ static int __init is_mmconf_reserved(check_reserved_t is_reserved,
 	return valid;
 }
 
-static int __devinit pci_mmcfg_check_reserved(struct pci_mmcfg_region *cfg,
+static int __ref pci_mmcfg_check_reserved(struct pci_mmcfg_region *cfg,
 					      int early)
 {
 	if (!early && !acpi_disabled) {
@@ -504,6 +511,15 @@ static int __devinit pci_mmcfg_check_reserved(struct pci_mmcfg_region *cfg,
 			       &cfg->res);
 	}
 
+	/*
+	 * e820_all_mapped() is marked as __init.
+	 * All entries from ACPI MCFG table have been checked at boot time.
+	 * For MCFG information constructed from hotpluggable host bridge's
+	 * _CBA method, just assume it's reserved.
+	 */
+	if (system_state == SYSTEM_RUNNING)
+		return 1;
+
 	/* Don't try to do this check unless configuration
 	   type 1 is available. how about type 2 ?*/
 	if (raw_pci_ops)
@@ -641,6 +657,7 @@ static void __init __pci_mmcfg_init(int early)
 		 * the architecture mmcfg setup could not initialize.
 		 */
 		pci_mmcfg_resources_inserted = 1;
+		pci_mmcfg_arch_init_failed = true;
 	}
 }
 
@@ -681,3 +698,98 @@ static int __init pci_mmcfg_late_insert_resources(void)
  * with other system resources.
  */
 late_initcall(pci_mmcfg_late_insert_resources);
+
+/* Add MMCFG information for host bridges */
+int __devinit pci_mmconfig_insert(uint16_t seg, uint8_t start, uint8_t end,
+				  phys_addr_t addr)
+{
+	int rc;
+	struct resource *tmp;
+	struct pci_mmcfg_region *cfg;
+
+	if (start > end)
+		return -EINVAL;
+
+	if (pci_mmcfg_arch_init_failed)
+		return -ENODEV;
+
+	mutex_lock(&pci_mmcfg_lock);
+	cfg = pci_mmconfig_lookup(seg, start);
+	if (cfg) {
+		if (cfg->end_bus < end)
+			printk(KERN_WARNING FW_BUG PREFIX
+			       "MMCONFIG entry for domain %04x [bus %02x-%02x] "
+			       "partially covers domain %04x [bus %02x-%02x]\n",
+			       seg, start, end,
+			       cfg->segment, cfg->start_bus, cfg->end_bus);
+		mutex_unlock(&pci_mmcfg_lock);
+		return -EEXIST;
+	}
+
+	if (!addr) {
+		mutex_unlock(&pci_mmcfg_lock);
+		return -EINVAL;
+	}
+
+	rc = -EBUSY;
+	cfg = pci_mmconfig_alloc(seg, start, end, addr);
+	if (cfg == NULL) {
+		rc = -ENOMEM;
+	} else if (!pci_mmcfg_check_reserved(cfg, 0)) {
+		printk(KERN_WARNING FW_BUG PREFIX
+		       "MMCONFIG entry for domain %04x [bus %02x-%02x] "
+		       "isn't reserved by firmware\n",
+			seg, start, end);
+	} else {
+		tmp = insert_resource_conflict(&iomem_resource, &cfg->res);
+		if (tmp) {
+			printk(KERN_WARNING PREFIX
+				"MMCONFIG resource %pR for domain %04x "
+				"[bus %02x-%02x] conflicts with resource "
+				"%s %pR\n",
+				&cfg->res, seg, start, end, tmp->name, tmp);
+		} else if (pci_mmcfg_arch_map(cfg)) {
+			printk(KERN_WARNING PREFIX
+				"MMCONFIG fail to map resource %pR for "
+				"domain %04x [bus %02x-%02x]\n",
+				&cfg->res, seg, start, end);
+		} else {
+			list_add_sorted(cfg);
+			cfg = NULL;
+			rc = 0;
+		}
+	}
+
+	if (cfg) {
+		if (cfg->res.parent)
+			release_resource(&cfg->res);
+		kfree(cfg);
+	}
+
+	mutex_unlock(&pci_mmcfg_lock);
+
+	return rc;
+}
+
+/* Delete MMCFG information for host bridges */
+int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end)
+{
+	struct pci_mmcfg_region *cfg;
+
+	mutex_lock(&pci_mmcfg_lock);
+	list_for_each_entry_rcu(cfg, &pci_mmcfg_list, list)
+		if (cfg->segment == seg && cfg->start_bus == start &&
+		    cfg->end_bus == end) {
+			list_del_rcu(&cfg->list);
+			synchronize_rcu();
+			pci_mmcfg_arch_unmap(cfg);
+			if (cfg->res.parent)
+				release_resource(&cfg->res);
+			mutex_unlock(&pci_mmcfg_lock);
+			kfree(cfg);
+			return 0;
+		}
+	mutex_unlock(&pci_mmcfg_lock);
+
+	return -ENOENT;
+}
-- 
1.7.1

[PATCH v6 6/9] PCI, ACPI: provide MCFG address for PCI host bridges

[Jiang Liu]

From: Jiang Liu <liuj97@gmail.com>

From: Jiang Liu <jiang.liu@huawei.com>

This patch provide MCFG address for PCI host bridges, which will
be used to support host bridge hotplug. It gets MCFG address
by evaluating _CBA method if available, or by scanning the ACPI
MCFG table.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 drivers/acpi/pci_root.c  |   12 ++++++++++++
 drivers/pci/pci-acpi.c   |   34 ++++++++++++++++++++++++++++++++++
 include/acpi/acnames.h   |    1 +
 include/acpi/acpi_bus.h  |    3 +++
 include/linux/pci-acpi.h |    5 +++++
 5 files changed, 55 insertions(+), 0 deletions(-)

diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index 7aff631..fc716cf 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -458,6 +458,7 @@ static int __devinit acpi_pci_root_add(struct acpi_device *device)
 	acpi_handle handle;
 	struct acpi_device *child;
 	u32 flags, base_flags;
+	int end_bus = -1;
 
 	root = kzalloc(sizeof(struct acpi_pci_root), GFP_KERNEL);
 	if (!root)
@@ -505,6 +506,17 @@ static int __devinit acpi_pci_root_add(struct acpi_device *device)
 	strcpy(acpi_device_class(device), ACPI_PCI_ROOT_CLASS);
 	device->driver_data = root;
 
+	root->mcfg_addr = acpi_pci_root_get_mcfg_addr(device->handle,
+		root->segment, (uint8_t)root->secondary.start, &end_bus);
+
+	/*
+	 * End bus number for MCFG may be less than root's subordinary
+	 * bus number with buggy BIOS implementation.
+	 */
+	if (end_bus < 0 || end_bus > root->secondary.end)
+		end_bus = root->secondary.end;
+	root->mcfg_end_bus = (uint8_t)end_bus;
+
 	/*
 	 * All supported architectures that use ACPI have support for
 	 * PCI domains, so we indicate this in _OSC support capabilities.
diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
index 0f150f2..9af71d2 100644
--- a/drivers/pci/pci-acpi.c
+++ b/drivers/pci/pci-acpi.c
@@ -162,6 +162,40 @@ acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
 	return remove_pm_notifier(dev, pci_acpi_wake_dev);
 }
 
+/* acpi_table_parse() is marked as __init, so cache MCFG info at boot time */
+int pci_acpi_mcfg_entries;
+struct acpi_mcfg_allocation *pci_acpi_mcfg_array;
+
+phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle, uint16_t seg,
+					uint8_t start, int *endp)
+{
+	int i, end_bus = -1;
+	acpi_status status = AE_NOT_EXIST;
+	unsigned long long mcfg_addr = 0;
+	struct acpi_mcfg_allocation *cfg;
+
+	if (handle)
+		status = acpi_evaluate_integer(handle, METHOD_NAME__CBA,
+					       NULL, &mcfg_addr);
+	if (ACPI_FAILURE(status) && pci_acpi_mcfg_entries &&
+	    pci_acpi_mcfg_array) {
+		mcfg_addr = 0;
+		cfg = pci_acpi_mcfg_array;
+		for (i = 0; i < pci_acpi_mcfg_entries; i++, cfg++)
+			if (seg == cfg->pci_segment &&
+			    start >= cfg->start_bus_number &&
+			    start <= cfg->end_bus_number) {
+				end_bus = cfg->end_bus_number;
+				mcfg_addr = cfg->address;
+				break;
+			}
+	}
+	if (endp)
+		*endp = end_bus;
+
+	return (phys_addr_t)mcfg_addr;
+}
+
 /*
  * _SxD returns the D-state with the highest power
  * (lowest D-state number) supported in the S-state "x".
diff --git a/include/acpi/acnames.h b/include/acpi/acnames.h
index 38f5088..99bda75 100644
--- a/include/acpi/acnames.h
+++ b/include/acpi/acnames.h
@@ -62,6 +62,7 @@
 #define METHOD_NAME__AEI        "_AEI"
 #define METHOD_NAME__PRW        "_PRW"
 #define METHOD_NAME__SRS        "_SRS"
+#define METHOD_NAME__CBA	"_CBA"
 
 /* Method names - these methods must appear at the namespace root */
 
diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
index f1c8ca6..8bc5229 100644
--- a/include/acpi/acpi_bus.h
+++ b/include/acpi/acpi_bus.h
@@ -370,6 +370,9 @@ struct acpi_pci_root {
 
 	u32 osc_support_set;	/* _OSC state of support bits */
 	u32 osc_control_set;	/* _OSC state of control bits */
+	uint8_t mcfg_end_bus;	/* End bus for MCFG may differ from
+				 * root's subordinary bus. */
+	phys_addr_t mcfg_addr;
 };
 
 /* helper */
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 4462350..0369149 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -11,12 +11,17 @@
 #include <linux/acpi.h>
 
 #ifdef CONFIG_ACPI
+extern int pci_acpi_mcfg_entries;
+extern struct acpi_mcfg_allocation *pci_acpi_mcfg_array;
+
 extern acpi_status pci_acpi_add_bus_pm_notifier(struct acpi_device *dev,
 						 struct pci_bus *pci_bus);
 extern acpi_status pci_acpi_remove_bus_pm_notifier(struct acpi_device *dev);
 extern acpi_status pci_acpi_add_pm_notifier(struct acpi_device *dev,
 					     struct pci_dev *pci_dev);
 extern acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev);
+extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle,
+			uint16_t seg, uint8_t start, int *endp);
 
 static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
 {
-- 
1.7.1

[PATCH v6 7/9] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Jiang Liu]

From: Jiang Liu <liuj97@gmail.com>

From: Jiang Liu <jiang.liu@huawei.com>

This patch enhances x86 arch specific code to update MMCFG information
when PCI host bridge hotplug event happens.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/include/asm/pci_x86.h |    1 +
 arch/x86/pci/acpi.c            |   71 ++++++++++++++++++++++++++++++++++++++++
 arch/x86/pci/mmconfig-shared.c |    7 +---
 arch/x86/pci/mmconfig_32.c     |    2 +-
 arch/x86/pci/mmconfig_64.c     |    2 +-
 5 files changed, 75 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index 1a3c12f..a50e783 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -100,6 +100,7 @@ struct pci_raw_ops {
 extern const struct pci_raw_ops *raw_pci_ops;
 extern const struct pci_raw_ops *raw_pci_ext_ops;
 
+extern const struct pci_raw_ops pci_mmcfg;
 extern const struct pci_raw_ops pci_direct_conf1;
 extern bool port_cf9_safe;
 
diff --git a/arch/x86/pci/acpi.c b/arch/x86/pci/acpi.c
index 2bb885a..ce29bdf 100644
--- a/arch/x86/pci/acpi.c
+++ b/arch/x86/pci/acpi.c
@@ -4,6 +4,7 @@
 #include <linux/irq.h>
 #include <linux/dmi.h>
 #include <linux/slab.h>
+#include <linux/pci-acpi.h>
 #include <asm/numa.h>
 #include <asm/pci_x86.h>
 
@@ -13,6 +14,12 @@ struct pci_root_info {
 	unsigned int res_num;
 	struct resource *res;
 	struct pci_sysdata sd;
+#ifdef	CONFIG_PCI_MMCONFIG
+	bool mcfg_added;
+	uint16_t segment;
+	uint8_t start_bus;
+	uint8_t end_bus;
+#endif
 };
 
 static bool pci_use_crs = true;
@@ -119,6 +126,63 @@ void __init pci_acpi_crs_quirks(void)
 	       pci_use_crs ? "nocrs" : "use_crs");
 }
 
+static int __devinit setup_mcfg_map(struct pci_root_info *info,
+				    uint16_t seg, uint8_t start, uint8_t end,
+				    phys_addr_t addr)
+{
+#ifdef	CONFIG_PCI_MMCONFIG
+	int result;
+
+	info->start_bus = start;
+	info->end_bus = end;
+	info->mcfg_added = false;
+
+	/* return success if MMCFG is not in use */
+	if (raw_pci_ext_ops && raw_pci_ext_ops != &pci_mmcfg)
+		return 0;
+
+	if (!(pci_probe & PCI_PROBE_MMCONF)) {
+		/* still could use raw_pci_ops for devices on segment 0 */
+		if (seg)
+			printk(KERN_WARNING
+				"MMCONFIG is disabled, can't access PCI device "
+				"configuration space on  %04x:%02x-%02x\n",
+				seg, start, end);
+		return 0;
+	}
+
+	result = pci_mmconfig_insert(seg, start, end, addr);
+	if (result == 0) {
+		/* enable MMCFG if it hasn't been enabled yet */
+		if (raw_pci_ext_ops == NULL)
+			raw_pci_ext_ops = &pci_mmcfg;
+		info->mcfg_added = true;
+	} else if (result != -EEXIST && addr) {
+		/*
+		 * Failure in adding MMCFG information is not fatal,
+		 * just can't access [extended] configuration space of
+		 * devices under this host bridge.
+		 */
+		printk(KERN_WARNING
+			"Fail to add MMCONFIG information for %04x:%02x-%02x\n",
+			seg, start, end);
+	}
+#endif
+
+	return 0;
+}
+
+static void teardown_mcfg_map(struct pci_root_info *info)
+{
+#ifdef	CONFIG_PCI_MMCONFIG
+	if (info->mcfg_added) {
+		pci_mmconfig_delete(info->segment, info->start_bus,
+				    info->end_bus);
+		info->mcfg_added = false;
+	}
+#endif
+}
+
 static acpi_status
 resource_to_addr(struct acpi_resource *resource,
 			struct acpi_resource_address64 *addr)
@@ -331,6 +395,8 @@ static void __release_pci_root_info(struct pci_root_info *info)
 
 	free_pci_root_info_res(info);
 
+	teardown_mcfg_map(info);
+
 	kfree(info);
 }
 static void release_pci_root_info(struct pci_host_bridge *bridge)
@@ -423,6 +489,11 @@ struct pci_bus * __devinit pci_acpi_scan_root(struct acpi_pci_root *root)
 		memcpy(bus->sysdata, sd, sizeof(*sd));
 		kfree(info);
 	} else {
+		if (root->mcfg_addr)
+			setup_mcfg_map(info, root->segment,
+				       (uint8_t)root->secondary.start,
+				       root->mcfg_end_bus, root->mcfg_addr);
+
 		probe_pci_root_info(info, device, busnum, domain);
 
 		/* insert busn res at first */
diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 8d0c287..4d0600b 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -707,7 +707,7 @@ int __devinit pci_mmconfig_insert(uint16_t seg, uint8_t start, uint8_t end,
 	struct resource *tmp;
 	struct pci_mmcfg_region *cfg;
 
-	if (start > end)
+	if (start > end || !addr)
 		return -EINVAL;
 
 	if (pci_mmcfg_arch_init_failed)
@@ -726,11 +726,6 @@ int __devinit pci_mmconfig_insert(uint16_t seg, uint8_t start, uint8_t end,
 		return -EEXIST;
 	}
 
-	if (!addr) {
-		mutex_unlock(&pci_mmcfg_lock);
-		return -EINVAL;
-	}
-
 	rc = -EBUSY;
 	cfg = pci_mmconfig_alloc(seg, start, end, addr);
 	if (cfg == NULL) {
diff --git a/arch/x86/pci/mmconfig_32.c b/arch/x86/pci/mmconfig_32.c
index a22785d..db63ac2 100644
--- a/arch/x86/pci/mmconfig_32.c
+++ b/arch/x86/pci/mmconfig_32.c
@@ -126,7 +126,7 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 	return 0;
 }
 
-static const struct pci_raw_ops pci_mmcfg = {
+const struct pci_raw_ops pci_mmcfg = {
 	.read =		pci_mmcfg_read,
 	.write =	pci_mmcfg_write,
 };
diff --git a/arch/x86/pci/mmconfig_64.c b/arch/x86/pci/mmconfig_64.c
index 4e05779..34c08dd 100644
--- a/arch/x86/pci/mmconfig_64.c
+++ b/arch/x86/pci/mmconfig_64.c
@@ -90,7 +90,7 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 	return 0;
 }
 
-static const struct pci_raw_ops pci_mmcfg = {
+const struct pci_raw_ops pci_mmcfg = {
 	.read =		pci_mmcfg_read,
 	.write =	pci_mmcfg_write,
 };
-- 
1.7.1

[PATCH v6 8/9] PCI, x86: add MMCFG information on demand

[Jiang Liu]

From: Jiang Liu <liuj97@gmail.com>

From: Jiang Liu <jiang.liu@huawei.com>

This patch changes mmconfig logic on x86 platforms to add MMCFG
information on demand instead of adding all MMCFG entries from
the ACPI MCFG table at boot time. So only MMCFG address ranges
used by active PCI host bridges will be actually mapped.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/include/asm/pci_x86.h |    5 ++++
 arch/x86/pci/legacy.c          |    1 +
 arch/x86/pci/mmconfig-shared.c |   51 ++++++++++++++++++++++++++++++++++++---
 3 files changed, 53 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index a50e783..dcf5d3e 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -141,6 +141,11 @@ extern void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg);
 extern int __devinit pci_mmconfig_insert(uint16_t seg, uint8_t start,
 					 uint8_t end, phys_addr_t addr);
 extern int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end);
+#ifdef	CONFIG_ACPI
+extern void pci_mmconfig_probe(uint8_t start);
+#else
+static inline void pci_mmconfig_probe(uint8_t start) { }
+#endif
 extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus);
 
 extern struct list_head pci_mmcfg_list;
diff --git a/arch/x86/pci/legacy.c b/arch/x86/pci/legacy.c
index a1df191..e9a2384 100644
--- a/arch/x86/pci/legacy.c
+++ b/arch/x86/pci/legacy.c
@@ -49,6 +49,7 @@ void __devinit pcibios_scan_specific_bus(int busn)
 		    l != 0x0000 && l != 0xffff) {
 			DBG("Found device at %02x:%02x [%04x]\n", busn, devfn, l);
 			printk(KERN_INFO "PCI: Discovered peer bus %02x\n", busn);
+			pci_mmconfig_probe(busn);
 			pci_scan_bus_on_node(busn, &pci_root_ops, node);
 			return;
 		}
diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 4d0600b..a7a2efd 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -19,6 +19,7 @@
 #include <linux/slab.h>
 #include <linux/mutex.h>
 #include <linux/rculist.h>
+#include <linux/pci-acpi.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 #include <asm/acpi.h>
@@ -600,6 +601,16 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
 			return -ENODEV;
 		}
 
+		/*
+		 * MMCFG information for host brideges will be added on demand
+		 * by pci_root driver if ACPI is enabled. But there are special
+		 * requirements for devices on segment 0, MMCFG information may
+		 * be needed for fixing hardware quirks and probing for hidden
+		 * buses.
+		 */
+		if (!acpi_disabled && cfg->pci_segment)
+			continue;
+
 		if (pci_mmconfig_add(cfg->pci_segment, cfg->start_bus_number,
 				   cfg->end_bus_number, cfg->address) == NULL) {
 			printk(KERN_WARNING PREFIX
@@ -609,6 +620,13 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
 		}
 	}
 
+	i = entries * sizeof(*cfg_table);
+	pci_acpi_mcfg_array = kmalloc(i, GFP_KERNEL);
+	if (pci_acpi_mcfg_array) {
+		memcpy(pci_acpi_mcfg_array, cfg_table, i);
+		pci_acpi_mcfg_entries = entries;
+	}
+
 	return 0;
 }
 
@@ -618,14 +636,14 @@ static void __init __pci_mmcfg_init(int early)
 	if ((pci_probe & PCI_PROBE_MMCONF) == 0)
 		return;
 
-	/* MMCONFIG already enabled */
-	if (!early && !(pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF))
-		return;
-
 	/* for late to exit */
 	if (known_bridge)
 		return;
 
+	/* MMCONFIG already enabled */
+	if (!early && !(pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF))
+		goto out;
+
 	if (early) {
 		if (pci_mmcfg_check_hostbridge())
 			known_bridge = 1;
@@ -659,6 +677,14 @@ static void __init __pci_mmcfg_init(int early)
 		pci_mmcfg_resources_inserted = 1;
 		pci_mmcfg_arch_init_failed = true;
 	}
+
+out:
+	/*
+	 * Free all MCFG entries if ACPI is enabled. MCFG information will
+	 * be added back on demand by the pci_root driver later.
+	 */
+	if (!early && !acpi_disabled && !known_bridge && pci_acpi_mcfg_array)
+		free_all_mmcfg();
 }
 
 void __init pci_mmcfg_early_init(void)
@@ -788,3 +814,20 @@ int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end)
 
 	return -ENOENT;
 }
+
+/* Probe MMCFG information for PCI bus blind probe */
+void __devinit pci_mmconfig_probe(uint8_t start)
+{
+	int end_bus, temp;
+	phys_addr_t addr;
+
+	addr = acpi_pci_root_get_mcfg_addr(NULL, 0, start, &end_bus);
+	if (addr && end_bus >= 0 && end_bus <= 255) {
+		for (temp = start + 1; temp <= end_bus; temp++)
+			if (pci_find_bus(0, temp))
+				break;
+
+		temp--;
+		pci_mmconfig_insert(0, start, (uint8_t)temp, addr);
+	}
+}
-- 
1.7.1

[PATCH v6 9/9] PCI, x86: simplify pci_mmcfg_late_insert_resources()

[Jiang Liu]

From: Jiang Liu <liuj97@gmail.com>

From: Jiang Liu <jiang.liu@huawei.com>

Reduce redundant code to simplify pci_mmcfg_late_insert_resources().

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/pci/mmconfig-shared.c |   32 +++++++-------------------------
 1 files changed, 7 insertions(+), 25 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index a7a2efd..01a775f 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -27,7 +27,6 @@
 #define PREFIX "PCI: "
 
 /* Indicate if the mmcfg resources have been placed into the resource table. */
-static int __initdata pci_mmcfg_resources_inserted;
 static bool pci_mmcfg_arch_init_failed;
 static DEFINE_MUTEX(pci_mmcfg_lock);
 
@@ -372,22 +371,6 @@ static int __init pci_mmcfg_check_hostbridge(void)
 	return !list_empty(&pci_mmcfg_list);
 }
 
-static void __init pci_mmcfg_insert_resources(void)
-{
-	struct pci_mmcfg_region *cfg;
-
-	/*
-	 * Insert resources for MMCFG items if the resource hasn't been
-	 * inserted by pci_mmconfig_insert() yet.
-	 */
-	list_for_each_entry(cfg, &pci_mmcfg_list, list)
-		if (!cfg->res.parent)
-			insert_resource(&iomem_resource, &cfg->res);
-
-	/* Mark that the resources have been inserted. */
-	pci_mmcfg_resources_inserted = 1;
-}
-
 static acpi_status __devinit check_mcfg_resource(struct acpi_resource *res,
 						 void *data)
 {
@@ -670,11 +653,7 @@ static void __init __pci_mmcfg_init(int early)
 	if (pci_mmcfg_arch_init())
 		pci_probe = (pci_probe & ~PCI_PROBE_MASK) | PCI_PROBE_MMCONF;
 	else {
-		/*
-		 * Signal not to attempt to insert mmcfg resources because
-		 * the architecture mmcfg setup could not initialize.
-		 */
-		pci_mmcfg_resources_inserted = 1;
+		free_all_mmcfg();
 		pci_mmcfg_arch_init_failed = true;
 	}
 
@@ -699,12 +678,13 @@ void __init pci_mmcfg_late_init(void)
 
 static int __init pci_mmcfg_late_insert_resources(void)
 {
+	struct pci_mmcfg_region *cfg;
+
 	/*
 	 * If resources are already inserted or we are not using MMCONFIG,
 	 * don't insert the resources.
 	 */
-	if ((pci_mmcfg_resources_inserted == 1) ||
-	    (pci_probe & PCI_PROBE_MMCONF) == 0 ||
+	if ((pci_probe & PCI_PROBE_MMCONF) == 0 ||
 	    list_empty(&pci_mmcfg_list))
 		return 1;
 
@@ -713,7 +693,9 @@ static int __init pci_mmcfg_late_insert_resources(void)
 	 * marked so it won't cause request errors when __request_region is
 	 * called.
 	 */
-	pci_mmcfg_insert_resources();
+	list_for_each_entry(cfg, &pci_mmcfg_list, list)
+		if (!cfg->res.parent)
+			insert_resource(&iomem_resource, &cfg->res);
 
 	return 0;
 }
-- 
1.7.1

Re: [PATCH v6 0/9] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Bjorn Helgaas]

On Wed, May 23, 2012 at 11:50:17AM +0800, Jiang Liu wrote:
> From: Jiang Liu <liuj97@gmail.com>
> 
> From: Jiang Liu <jiang.liu@huawei.com>
> 
> This patchset enhance pci_root driver to update MMCFG information when
> hot-plugging PCI root bridges. It applies to 
> git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git pci/next-3.5
> 
> -v2: split into smaller patches and skip updating MMCFG information when
>      MMCFG is disabled
> -v3: add mmconf_added to simply free path, also make pci_mmconfig_insert()
>      to process extra exist case --- By Yinghai
> -v4: tune arch_acpi_pci_root_add() to handle a corner case raised by Kenji
> -v5: address review comments from Bjorn and Taku, also better handle corner
>      cases in arch_acpi_pci_root_add()
> -v6: get rid of arch_acpi_pci_root_xxx() by using existing hooks
>      add MCFG information for host bridges on demand
>      more corner cases clear up
>      correctly handle condition compilation
>      fix section mismatch issues
>      fix a issue reported by Taku about a BIOS bug
> 
> The first 4 patches in series is the same with v5.

I tried this on one of my machines.  Before your patches:

    ACPI: bus type pci registered
    PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)
    PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820
    ...
    ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7e])

After:

    ACPI: bus type pci registered
1)  PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)
    PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820
    ...
    ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7e])
2)  PCI: MMCONFIG for domain 0000 [bus 00-7e] at [mem 0x80000000-0x87efffff] (base 0x80000000)
3)  [Firmware Bug]: PCI: MMCONFIG at [mem 0x80000000-0x87efffff] not reserved in ACPI motherboard resources
4)  PCI: MMCONFIG at [mem 0x80000000-0x87efffff] reserved in E820

I really like the MMCONFIG info being associated with the host bridge in
dmesg, and I really like it being split out in /proc/iomem.  That's much
cleaner than it used to be.  MMCONFIG is defined by the PCIe spec, and
there are non-x86 architectures that use the same mechanism, so I can
imagine someday (not now!) making it more generic.

1): Do we really need this early MCFG scan?  Conceptually it seems like we
don't need to look at MCFG until we add a host bridge or do a blind probe.

2) and 4): These should be dev_info() with the host bridge device, e.g.,

    pci_root PNP0A08:00: MMCONFIG at [mem 0x80000000-0x87efffff]

Now that we can associate it with the bridge, the bus number range is
redundant.  It might be useful to know whether the MMCONFIG range came from
_CBA or MCFG, if that's easy to do.

Here's I'm thinking: right now we have some message confusion -- the arch
prints window info, e.g., "pci_root PNP0A08:00: host bridge window [io
0x0000-0x03af]", and then the PCI core prints the same thing but associated
with the bus: "pci_bus 0000:00: root bus resource [io  0x0000-0x03af]".
I'd like this to become more uniform, and have only the core print it, but
using the host bridge device, not the bus.

3): Something seems wrong here -- this "bug" message didn't appear before,
so I don't think it should now.  I haven't tried to chase this down.

Bjorn

Re: [PATCH v6 0/9] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Bjorn Helgaas]

On Thu, May 24, 2012 at 12:26 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>    ACPI: bus type pci registered
> 1)  PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)
>    PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820
>    ...

> 1): Do we really need this early MCFG scan?  Conceptually it seems like we
> don't need to look at MCFG until we add a host bridge or do a blind probe.

I looked at mmconfig-shared.c again.  I had forgotten what a total
pile of crap it is.  I haven't seen so many special cases and
exceptions in a long time.

I think we probably *could* drop this early scan by converting the
pci_mmcfg_check_hostbridge() stuff into quirks (i.e., enumerate
devices without MMCONFIG and have quirks add MMCONFIG space when their
devices are found), but that would be major surgery that isn't
relevant to your project.  So just ignore my question about this.

Bjorn

Re: [PATCH v6 0/9] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Jiang Liu]

On 2012-5-25 2:26, Bjorn Helgaas wrote:
> On Wed, May 23, 2012 at 11:50:17AM +0800, Jiang Liu wrote:
>> From: Jiang Liu<liuj97@gmail.com>
>>
>> From: Jiang Liu<jiang.liu@huawei.com>
>>
>> This patchset enhance pci_root driver to update MMCFG information when
>> hot-plugging PCI root bridges. It applies to
>> git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git pci/next-3.5
>>
>> -v2: split into smaller patches and skip updating MMCFG information when
>>       MMCFG is disabled
>> -v3: add mmconf_added to simply free path, also make pci_mmconfig_insert()
>>       to process extra exist case --- By Yinghai
>> -v4: tune arch_acpi_pci_root_add() to handle a corner case raised by Kenji
>> -v5: address review comments from Bjorn and Taku, also better handle corner
>>       cases in arch_acpi_pci_root_add()
>> -v6: get rid of arch_acpi_pci_root_xxx() by using existing hooks
>>       add MCFG information for host bridges on demand
>>       more corner cases clear up
>>       correctly handle condition compilation
>>       fix section mismatch issues
>>       fix a issue reported by Taku about a BIOS bug
>>
>> The first 4 patches in series is the same with v5.
>
> I tried this on one of my machines.  Before your patches:
>
>      ACPI: bus type pci registered
>      PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)
>      PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820
>      ...
>      ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7e])
>
> After:
>
>      ACPI: bus type pci registered
> 1)  PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)
>      PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820
>      ...
>      ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7e])
> 2)  PCI: MMCONFIG for domain 0000 [bus 00-7e] at [mem 0x80000000-0x87efffff] (base 0x80000000)
> 3)  [Firmware Bug]: PCI: MMCONFIG at [mem 0x80000000-0x87efffff] not reserved in ACPI motherboard resources
> 4)  PCI: MMCONFIG at [mem 0x80000000-0x87efffff] reserved in E820
>
> I really like the MMCONFIG info being associated with the host bridge in
> dmesg, and I really like it being split out in /proc/iomem.  That's much
> cleaner than it used to be.  MMCONFIG is defined by the PCIe spec, and
> there are non-x86 architectures that use the same mechanism, so I can
> imagine someday (not now!) making it more generic.
>
> 1): Do we really need this early MCFG scan?  Conceptually it seems like we
> don't need to look at MCFG until we add a host bridge or do a blind probe.
After digging the log messages, I found the early probing logic is
needed for two cases:
1) CONFIG_PCI_BIOS and CONFIG_PCI_DIRECT are disabled or not supported
by the platform, and MMCONF is the only method to access configuration
space. So need to support MMCONF as early as possible because ACPI
initialization may need to access PCI configuration space.

2) ACPI is disabled by "acpi=off" boot option or due to errors when 
initializing the ACPI subsystem, so pci_mmcfg_late_init() won't be called.

So it seems we do need the early probe logic.

>
> 2) and 4): These should be dev_info() with the host bridge device, e.g.,
>
>      pci_root PNP0A08:00: MMCONFIG at [mem 0x80000000-0x87efffff]
>
> Now that we can associate it with the bridge, the bus number range is
> redundant.  It might be useful to know whether the MMCONFIG range came from
> _CBA or MCFG, if that's easy to do.
>
> Here's I'm thinking: right now we have some message confusion -- the arch
> prints window info, e.g., "pci_root PNP0A08:00: host bridge window [io
> 0x0000-0x03af]", and then the PCI core prints the same thing but associated
> with the bus: "pci_bus 0000:00: root bus resource [io  0x0000-0x03af]".
> I'd like this to become more uniform, and have only the core print it, but
> using the host bridge device, not the bus.
OK, will change the log messages according to your suggestions.

>
> 3): Something seems wrong here -- this "bug" message didn't appear before,
> so I don't think it should now.  I haven't tried to chase this down.
__pci_mmcfg_init() will be called by pci_mmcfg_early_init() and
pci_mmcfg_late_init(). When it's called by pci_mmcfg_early_init(),
the ACPI subsystem hasn't been initialized yet, so
pci_mmcfg_reject_broken() doesn't check whether the resource is
reserved by ACPI motherboard devices. If we pass "pci=noearly"
option to the kernel to disable early probe, it will generate
the warning message on the same platform.

According to commit 7752d5cfe3d11ca0bb9c673ec38bd78ba6578f8e log
message, "The PCI Express firmware spec apparently tells BIOS
developers that reservation in ACPI is required and E820 reservation
is optional", it would be better for us to emit this warning message.
Any suggestions here?

Re: [PATCH v6 0/9] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Taku Izumi]

Hi Jiang,

 After applied your patch, the iomem tree structure a bit changed
 on my machine. The entry of "f0000000-f3ffffff : pnp 00:02" has gone.
 This is because the order of insert_resource has changed, in my mind.
 After applied your patch, "PCI MMCONFIG 0000" is inserted before
 "pnp 00:02", so inserting "pnp 00:02" failed.

* /proc/iomem (before)
...
efee0000-efeecfff : ACPI Tables
efeed000-efefffff : ACPI Non-volatile Storage
eff00000-f3ffffff : reserved
  f0000000-f3ffffff : pnp 00:02   <---
    f0000000-f13fffff : PCI MMCONFIG 0000 [bus 00-13]
f4000000-f41fffff : PCI Bus 0000:01
  f4000000-f41fffff : PCI Bus 0000:02
    f4000000-f40fffff : 0000:02:05.0
    f4100000-f413ffff : 0000:02:09.0
    f4140000-f417ffff : 0000:02:09.1
....

* /proc/iomem (after)
...
efee0000-efeecfff : ACPI Tables
efeed000-efefffff : ACPI Non-volatile Storage
eff00000-f3ffffff : reserved
  f0000000-f13fffff : PCI MMCONFIG 0000 [bus 00-13]
f4000000-f41fffff : PCI Bus 0000:01
  f4000000-f41fffff : PCI Bus 0000:02
    f4000000-f40fffff : 0000:02:05.0
    f4100000-f413ffff : 0000:02:09.0
    f4140000-f417ffff : 0000:02:09.1
...

Best regards,
Taku Izumi

Re: [PATCH v6 0/9] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Jiang Liu]

Hi Taku,
	Thanks for testing, will fix this issue in next version.

On 05/25/2012 06:22 PM, Taku Izumi wrote:
> efee0000-efeecfff : ACPI Tables