[PATCH v11 00/15] PCI: ARM64 ECAM quirks

[PATCH v11 00/15] PCI: ARM64 ECAM quirks

[Bjorn Helgaas]

Here's another try.  The biggest change is to add the APM X-Gene quirks.

These are on my pci/ecam branch.  Please test and report any issues.
Please also collect dmesg and /proc/iomem contents and I'll try to
summarize the current ACPI firmware situation on ARM64 and update the
writeup with recommendations for future platforms.

Changes from v10 to v11:

- Discard "consumer" resources from PNP0A03 host bridge windows (arm64
  only).  This allows using consumer PNP0A03 descriptors (rather than the
  PNP0C02 workaround used on x86) for bridge register space.

- Add APM X-Gene MCFG quirks.

- If ACPI and quirks are enabled, always build ThunderX, X-Gene, and
  HiSilicon drivers on ARM64.  Add internal ifdefs so we only compile the
  parts necessary for ACPI (the ECAM init and accessors).

- s/node/seg/ in ThunderX MCFG quirk macros.

- Move ECAM check for ACPI device reservation from pci_ecam_create() to
  that arm64-specific pci_acpi_setup_ecam_mapping().

- Add local "struct device *dev" pointers for brevity.

- Miscellaneous build fixes.

---

Bjorn Helgaas (6):
      ACPI: Add acpi_resource_consumer() to find device that claims a resource
      x86/PCI: Use acpi_resource_consumer() to search ACPI namespace for MMCFG
      arm64: PCI: Add local struct device pointers
      arm64: PCI: Search ACPI namespace to ensure ECAM space is reserved
      arm64: PCI: Exclude ACPI "consumer" resources from host bridge windows
      PCI: thunder-pem: Factor out resource lookup

Christopher Covington (1):
      PCI: Add MCFG quirks for Qualcomm QDF2432 host controller

Dongdong Liu (2):
      PCI/ACPI: Provide acpi_get_rc_resources() for ARM64 platform
      PCI: Add MCFG quirks for HiSilicon Hip05/06/07 host controllers

Duc Dang (1):
      PCI: Add MCFG quirks for X-Gene host controller

Tomasz Nowicki (5):
      arm64: PCI: Manage controller-specific data on per-controller basis
      PCI/ACPI: Extend pci_mcfg_lookup() to return ECAM config accessors
      PCI/ACPI: Check for platform-specific MCFG quirks
      PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller
      PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller


 arch/arm64/kernel/pci.c             |   68 ++++++++-----
 arch/x86/pci/mmconfig-shared.c      |   69 ++-----------
 drivers/acpi/pci_mcfg.c             |  190 ++++++++++++++++++++++++++++++++++-
 drivers/acpi/resource.c             |   57 +++++++++++
 drivers/pci/ecam.c                  |   12 ++
 drivers/pci/host/Kconfig            |   10 +-
 drivers/pci/host/Makefile           |    8 +
 drivers/pci/host/pci-thunder-ecam.c |    9 +-
 drivers/pci/host/pci-thunder-pem.c  |   94 +++++++++++++----
 drivers/pci/host/pci-xgene.c        |  126 ++++++++++++++++++++++-
 drivers/pci/host/pcie-hisi.c        |  101 +++++++++++++++++++
 drivers/pci/pci-acpi.c              |   76 ++++++++++++++
 drivers/pci/pci.h                   |    5 +
 include/linux/acpi.h                |    7 +
 include/linux/pci-acpi.h            |    4 +
 include/linux/pci-ecam.h            |    9 ++
 16 files changed, 718 insertions(+), 127 deletions(-)

[PATCH v11 01/15] ACPI: Add acpi_resource_consumer() to find device that claims a resource

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

Add acpi_resource_consumer().  This takes a struct resource and searches
the ACPI namespace for a device whose current resource settings (_CRS)
includes the resource.  It returns the device if it exists, or NULL if no
device uses the resource.

If more than one device uses the resource (this may happen in the case of
bridges), acpi_resource_consumer() returns the first one found by
acpi_get_devices() in its modified depth-first walk of the namespace.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
---
 drivers/acpi/resource.c |   57 +++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/acpi.h    |    7 ++++++
 2 files changed, 64 insertions(+)

diff --git a/drivers/acpi/resource.c b/drivers/acpi/resource.c
index 56241eb..cb57962 100644
--- a/drivers/acpi/resource.c
+++ b/drivers/acpi/resource.c
@@ -664,3 +664,60 @@ int acpi_dev_filter_resource_type(struct acpi_resource *ares,
 	return (type & types) ? 0 : 1;
 }
 EXPORT_SYMBOL_GPL(acpi_dev_filter_resource_type);
+
+static int acpi_dev_consumes_res(struct acpi_device *adev, struct resource *res)
+{
+	struct list_head resource_list;
+	struct resource_entry *rentry;
+	int ret, found = 0;
+
+	INIT_LIST_HEAD(&resource_list);
+	ret = acpi_dev_get_resources(adev, &resource_list, NULL, NULL);
+	if (ret < 0)
+		return 0;
+
+	list_for_each_entry(rentry, &resource_list, node) {
+		if (resource_contains(rentry->res, res)) {
+			found = 1;
+			break;
+		}
+
+	}
+
+	acpi_dev_free_resource_list(&resource_list);
+	return found;
+}
+
+static acpi_status acpi_res_consumer_cb(acpi_handle handle, u32 depth,
+					 void *context, void **ret)
+{
+	struct resource *res = context;
+	struct acpi_device **consumer = (struct acpi_device **) ret;
+	struct acpi_device *adev;
+
+	if (acpi_bus_get_device(handle, &adev))
+		return AE_OK;
+
+	if (acpi_dev_consumes_res(adev, res)) {
+		*consumer = adev;
+		return AE_CTRL_TERMINATE;
+	}
+
+	return AE_OK;
+}
+
+/**
+ * acpi_resource_consumer - Find the ACPI device that consumes @res.
+ * @res: Resource to search for.
+ *
+ * Search the current resource settings (_CRS) of every ACPI device node
+ * for @res.  If we find an ACPI device whose _CRS includes @res, return
+ * it.  Otherwise, return NULL.
+ */
+struct acpi_device *acpi_resource_consumer(struct resource *res)
+{
+	struct acpi_device *consumer = NULL;
+
+	acpi_get_devices(NULL, acpi_res_consumer_cb, res, (void **) &consumer);
+	return consumer;
+}
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index ddbeda6..b00ad73 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -419,6 +419,8 @@ static inline int acpi_dev_filter_resource_type_cb(struct acpi_resource *ares,
 	return acpi_dev_filter_resource_type(ares, (unsigned long)arg);
 }
 
+struct acpi_device *acpi_resource_consumer(struct resource *res);
+
 int acpi_check_resource_conflict(const struct resource *res);
 
 int acpi_check_region(resource_size_t start, resource_size_t n,
@@ -762,6 +764,11 @@ static inline int acpi_reconfig_notifier_unregister(struct notifier_block *nb)
 	return -EINVAL;
 }
 
+static inline struct acpi_device *acpi_resource_consumer(struct resource *res)
+{
+	return NULL;
+}
+
 #endif	/* !CONFIG_ACPI */
 
 #ifdef CONFIG_ACPI_HOTPLUG_IOAPIC

[PATCH v11 02/15] x86/PCI: Use acpi_resource_consumer() to search ACPI namespace for MMCFG

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

The static MCFG table tells us the base of MMCFG space, but it does not
reserve the space -- the reservation should be done via a device in the
ACPI namespace whose _CRS includes the MMCFG region.

Replace find_mboard_resource(), which already searches for such a device,
with the new acpi_resource_consumer() interface that does essentially the
same thing.

acpi_resource_consumer() is not as strict as find_mboard_resource() was:
find_mboard_resource() only looks at PNP0C01 and PNP0C02 devices and the
following _CRS descriptor types:

  ACPI_RESOURCE_TYPE_FIXED_MEMORY32
  ACPI_RESOURCE_TYPE_ADDRESS32
  ACPI_RESOURCE_TYPE_ADDRESS64

but acpi_resource_consumer() looks at *all* devices in the namespace and
the following descriptor types:

  ACPI_RESOURCE_TYPE_MEMORY24
  ACPI_RESOURCE_TYPE_MEMORY32
  ACPI_RESOURCE_TYPE_FIXED_MEMORY32
  ACPI_RESOURCE_TYPE_ADDRESS16
  ACPI_RESOURCE_TYPE_ADDRESS32
  ACPI_RESOURCE_TYPE_ADDRESS64
  ACPI_RESOURCE_TYPE_EXTENDED_ADDRESS64

I think it is correct to accept the larger set of descriptor types.

Note 2 to Table 4-2 of the PCI Firmware spec r3.2 does suggest that the
MMCFG region should be declared by a PNP0C02 device.  I don't know why it
calls out PNP0C02 specifically, unless it's related to the fact that the
Consumer/Producer issues make it hard to use the PNP0A03 device itself.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/x86/pci/mmconfig-shared.c |   69 +++++-----------------------------------
 1 file changed, 9 insertions(+), 60 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index dd30b7e..f62ffe7 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -373,71 +373,20 @@ static int __init pci_mmcfg_check_hostbridge(void)
 	return !list_empty(&pci_mmcfg_list);
 }
 
-static acpi_status check_mcfg_resource(struct acpi_resource *res, void *data)
-{
-	struct resource *mcfg_res = data;
-	struct acpi_resource_address64 address;
-	acpi_status status;
-
-	if (res->type == ACPI_RESOURCE_TYPE_FIXED_MEMORY32) {
-		struct acpi_resource_fixed_memory32 *fixmem32 =
-			&res->data.fixed_memory32;
-		if (!fixmem32)
-			return AE_OK;
-		if ((mcfg_res->start >= fixmem32->address) &&
-		    (mcfg_res->end < (fixmem32->address +
-				      fixmem32->address_length))) {
-			mcfg_res->flags = 1;
-			return AE_CTRL_TERMINATE;
-		}
-	}
-	if ((res->type != ACPI_RESOURCE_TYPE_ADDRESS32) &&
-	    (res->type != ACPI_RESOURCE_TYPE_ADDRESS64))
-		return AE_OK;
-
-	status = acpi_resource_to_address64(res, &address);
-	if (ACPI_FAILURE(status) ||
-	   (address.address.address_length <= 0) ||
-	   (address.resource_type != ACPI_MEMORY_RANGE))
-		return AE_OK;
-
-	if ((mcfg_res->start >= address.address.minimum) &&
-	    (mcfg_res->end < (address.address.minimum + address.address.address_length))) {
-		mcfg_res->flags = 1;
-		return AE_CTRL_TERMINATE;
-	}
-	return AE_OK;
-}
-
-static acpi_status find_mboard_resource(acpi_handle handle, u32 lvl,
-					void *context, void **rv)
-{
-	struct resource *mcfg_res = context;
-
-	acpi_walk_resources(handle, METHOD_NAME__CRS,
-			    check_mcfg_resource, context);
-
-	if (mcfg_res->flags)
-		return AE_CTRL_TERMINATE;
-
-	return AE_OK;
-}
-
 static int is_acpi_reserved(u64 start, u64 end, unsigned not_used)
 {
 	struct resource mcfg_res;
+	struct acpi_device *adev;
 
 	mcfg_res.start = start;
 	mcfg_res.end = end - 1;
-	mcfg_res.flags = 0;
+	mcfg_res.flags = IORESOURCE_MEM;
 
-	acpi_get_devices("PNP0C01", find_mboard_resource, &mcfg_res, NULL);
-
-	if (!mcfg_res.flags)
-		acpi_get_devices("PNP0C02", find_mboard_resource, &mcfg_res,
-				 NULL);
+	adev = acpi_resource_consumer(&mcfg_res);
+	if (adev)
+		return 1;
 
-	return mcfg_res.flags;
+	return 0;
 }
 
 typedef int (*check_reserved_t)(u64 start, u64 end, unsigned type);
@@ -450,7 +399,7 @@ static int __ref is_mmconf_reserved(check_reserved_t is_reserved,
 	u64 size = resource_size(&cfg->res);
 	u64 old_size = size;
 	int num_buses;
-	char *method = with_e820 ? "E820" : "ACPI motherboard resources";
+	char *method = with_e820 ? "E820" : "ACPI namespace";
 
 	while (!is_reserved(addr, addr + size, E820_RESERVED)) {
 		size >>= 1;
@@ -504,12 +453,12 @@ static int __ref pci_mmcfg_check_reserved(struct device *dev,
 		if (dev)
 			dev_info(dev, FW_INFO
 				 "MMCONFIG at %pR not reserved in "
-				 "ACPI motherboard resources\n",
+				 "ACPI namespace\n",
 				 &cfg->res);
 		else
 			pr_info(FW_INFO PREFIX
 			       "MMCONFIG at %pR not reserved in "
-			       "ACPI motherboard resources\n",
+			       "ACPI namespace\n",
 			       &cfg->res);
 	}
 

[PATCH v11 03/15] arm64: PCI: Add local struct device pointers

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

Use a local "struct device *dev" for brevity.  No functional change
intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/arm64/kernel/pci.c |   11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index acf3872..8183c98 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -121,6 +121,7 @@ int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
 static struct pci_config_window *
 pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
 {
+	struct device *dev = &root->device->dev;
 	struct resource *bus_res = &root->secondary;
 	u16 seg = root->segment;
 	struct pci_config_window *cfg;
@@ -132,8 +133,7 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
 		root->mcfg_addr = pci_mcfg_lookup(seg, bus_res);
 
 	if (!root->mcfg_addr) {
-		dev_err(&root->device->dev, "%04x:%pR ECAM region not found\n",
-			seg, bus_res);
+		dev_err(dev, "%04x:%pR ECAM region not found\n", seg, bus_res);
 		return NULL;
 	}
 
@@ -141,11 +141,10 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
 	cfgres.start = root->mcfg_addr + bus_res->start * bsz;
 	cfgres.end = cfgres.start + resource_size(bus_res) * bsz - 1;
 	cfgres.flags = IORESOURCE_MEM;
-	cfg = pci_ecam_create(&root->device->dev, &cfgres, bus_res,
-			      &pci_generic_ecam_ops);
+	cfg = pci_ecam_create(dev, &cfgres, bus_res, &pci_generic_ecam_ops);
 	if (IS_ERR(cfg)) {
-		dev_err(&root->device->dev, "%04x:%pR error %ld mapping ECAM\n",
-			seg, bus_res, PTR_ERR(cfg));
+		dev_err(dev, "%04x:%pR error %ld mapping ECAM\n", seg, bus_res,
+			PTR_ERR(cfg));
 		return NULL;
 	}
 

Re: [PATCH v11 03/15] arm64: PCI: Add local struct device pointers

[Lorenzo Pieralisi]

On Mon, Dec 05, 2016 at 05:25:42PM -0600, Bjorn Helgaas wrote:
> From: Bjorn Helgaas <bhelgaas@google.com>
> 
> Use a local "struct device *dev" for brevity.  No functional change
> intended.
> 
> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> ---
>  arch/arm64/kernel/pci.c |   11 +++++------
>  1 file changed, 5 insertions(+), 6 deletions(-)

Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
> index acf3872..8183c98 100644
> --- a/arch/arm64/kernel/pci.c
> +++ b/arch/arm64/kernel/pci.c
> @@ -121,6 +121,7 @@ int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
>  static struct pci_config_window *
>  pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
>  {
> +	struct device *dev = &root->device->dev;
>  	struct resource *bus_res = &root->secondary;
>  	u16 seg = root->segment;
>  	struct pci_config_window *cfg;
> @@ -132,8 +133,7 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
>  		root->mcfg_addr = pci_mcfg_lookup(seg, bus_res);
>  
>  	if (!root->mcfg_addr) {
> -		dev_err(&root->device->dev, "%04x:%pR ECAM region not found\n",
> -			seg, bus_res);
> +		dev_err(dev, "%04x:%pR ECAM region not found\n", seg, bus_res);
>  		return NULL;
>  	}
>  
> @@ -141,11 +141,10 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
>  	cfgres.start = root->mcfg_addr + bus_res->start * bsz;
>  	cfgres.end = cfgres.start + resource_size(bus_res) * bsz - 1;
>  	cfgres.flags = IORESOURCE_MEM;
> -	cfg = pci_ecam_create(&root->device->dev, &cfgres, bus_res,
> -			      &pci_generic_ecam_ops);
> +	cfg = pci_ecam_create(dev, &cfgres, bus_res, &pci_generic_ecam_ops);
>  	if (IS_ERR(cfg)) {
> -		dev_err(&root->device->dev, "%04x:%pR error %ld mapping ECAM\n",
> -			seg, bus_res, PTR_ERR(cfg));
> +		dev_err(dev, "%04x:%pR error %ld mapping ECAM\n", seg, bus_res,
> +			PTR_ERR(cfg));
>  		return NULL;
>  	}
>  
> 

[PATCH v11 04/15] arm64: PCI: Search ACPI namespace to ensure ECAM space is reserved

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

The static MCFG table tells us the base of ECAM space, but it does not
reserve the space -- the reservation should be done via a device in the
ACPI namespace whose _CRS includes the ECAM region.

Use acpi_resource_consumer() to check whether the ECAM space is reserved by
an ACPI namespace device.  If it is, emit a message showing which device
reserves it.  If not, emit a "[Firmware Bug]" warning.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/arm64/kernel/pci.c |   13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 8183c98..7909f59 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -124,8 +124,9 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
 	struct device *dev = &root->device->dev;
 	struct resource *bus_res = &root->secondary;
 	u16 seg = root->segment;
-	struct pci_config_window *cfg;
 	struct resource cfgres;
+	struct acpi_device *adev;
+	struct pci_config_window *cfg;
 	unsigned int bsz;
 
 	/* Use address from _CBA if present, otherwise lookup MCFG */
@@ -141,6 +142,16 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
 	cfgres.start = root->mcfg_addr + bus_res->start * bsz;
 	cfgres.end = cfgres.start + resource_size(bus_res) * bsz - 1;
 	cfgres.flags = IORESOURCE_MEM;
+
+	adev = acpi_resource_consumer(&cfgres);
+	if (adev)
+		dev_info(dev, "ECAM area %pR reserved by %s\n", &cfgres,
+			 dev_name(&adev->dev));
+	else
+		dev_warn(dev, FW_BUG "ECAM area %pR not reserved in ACPI namespace\n",
+			 &cfgres);
+
+
 	cfg = pci_ecam_create(dev, &cfgres, bus_res, &pci_generic_ecam_ops);
 	if (IS_ERR(cfg)) {
 		dev_err(dev, "%04x:%pR error %ld mapping ECAM\n", seg, bus_res,

Re: [PATCH v11 04/15] arm64: PCI: Search ACPI namespace to ensure ECAM space is reserved

[Lorenzo Pieralisi]

On Mon, Dec 05, 2016 at 05:25:50PM -0600, Bjorn Helgaas wrote:
> From: Bjorn Helgaas <bhelgaas@google.com>
> 
> The static MCFG table tells us the base of ECAM space, but it does not
> reserve the space -- the reservation should be done via a device in the
> ACPI namespace whose _CRS includes the ECAM region.
> 
> Use acpi_resource_consumer() to check whether the ECAM space is reserved by
> an ACPI namespace device.  If it is, emit a message showing which device
> reserves it.  If not, emit a "[Firmware Bug]" warning.
> 
> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> ---
>  arch/arm64/kernel/pci.c |   13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
> index 8183c98..7909f59 100644
> --- a/arch/arm64/kernel/pci.c
> +++ b/arch/arm64/kernel/pci.c
> @@ -124,8 +124,9 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
>  	struct device *dev = &root->device->dev;
>  	struct resource *bus_res = &root->secondary;
>  	u16 seg = root->segment;
> -	struct pci_config_window *cfg;
>  	struct resource cfgres;
> +	struct acpi_device *adev;
> +	struct pci_config_window *cfg;
>  	unsigned int bsz;
>  
>  	/* Use address from _CBA if present, otherwise lookup MCFG */
> @@ -141,6 +142,16 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
>  	cfgres.start = root->mcfg_addr + bus_res->start * bsz;
>  	cfgres.end = cfgres.start + resource_size(bus_res) * bsz - 1;
>  	cfgres.flags = IORESOURCE_MEM;
> +
> +	adev = acpi_resource_consumer(&cfgres);
> +	if (adev)

I think that currently:

if (unlikely(adev))

would be more appropriate. Joking (not really ;-))

> +		dev_info(dev, "ECAM area %pR reserved by %s\n", &cfgres,
> +			 dev_name(&adev->dev));
> +	else
> +		dev_warn(dev, FW_BUG "ECAM area %pR not reserved in ACPI namespace\n",
> +			 &cfgres);

This log is going to be rather popular.

> +
> +

Nit: One too many empty line.

Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

>  	cfg = pci_ecam_create(dev, &cfgres, bus_res, &pci_generic_ecam_ops);
>  	if (IS_ERR(cfg)) {
>  		dev_err(dev, "%04x:%pR error %ld mapping ECAM\n", seg, bus_res,
> 

[PATCH v11 05/15] arm64: PCI: Manage controller-specific data on per-controller basis

[Bjorn Helgaas]

From: Tomasz Nowicki <tn@semihalf.com>

Currently we use one shared global acpi_pci_root_ops structure to keep
controller-specific ops. We pass its pointer to acpi_pci_root_create() and
associate it with a host bridge instance for good.  Such a design implies
serious drawback. Any potential manipulation on the single system-wide
acpi_pci_root_ops leads to kernel crash. The structure content is not
really changing even across multiple host bridges creation; thus it was not
an issue so far.

In preparation for adding ECAM quirks mechanism (where controller-specific
PCI ops may be different for each host bridge) allocate new
acpi_pci_root_ops and fill in with data for each bridge. Now it is safe to
have different controller-specific info. As a consequence free
acpi_pci_root_ops when host bridge is released.

No functional changes in this patch.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/arm64/kernel/pci.c |   17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 7909f59..1eb42ba 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -169,33 +169,36 @@ static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
 
 	ri = container_of(ci, struct acpi_pci_generic_root_info, common);
 	pci_ecam_free(ri->cfg);
+	kfree(ci->ops);
 	kfree(ri);
 }
 
-static struct acpi_pci_root_ops acpi_pci_root_ops = {
-	.release_info = pci_acpi_generic_release_info,
-};
-
 /* Interface called from ACPI code to setup PCI host controller */
 struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
 {
 	int node = acpi_get_node(root->device->handle);
 	struct acpi_pci_generic_root_info *ri;
 	struct pci_bus *bus, *child;
+	struct acpi_pci_root_ops *root_ops;
 
 	ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
 	if (!ri)
 		return NULL;
 
+	root_ops = kzalloc_node(sizeof(*root_ops), GFP_KERNEL, node);
+	if (!root_ops)
+		return NULL;
+
 	ri->cfg = pci_acpi_setup_ecam_mapping(root);
 	if (!ri->cfg) {
 		kfree(ri);
+		kfree(root_ops);
 		return NULL;
 	}
 
-	acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
-	bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
-				   ri->cfg);
+	root_ops->release_info = pci_acpi_generic_release_info;
+	root_ops->pci_ops = &ri->cfg->ops->pci_ops;
+	bus = acpi_pci_root_create(root, root_ops, &ri->common, ri->cfg);
 	if (!bus)
 		return NULL;
 

Re: [PATCH v11 05/15] arm64: PCI: Manage controller-specific data on per-controller basis

[Lorenzo Pieralisi]

On Mon, Dec 05, 2016 at 05:25:57PM -0600, Bjorn Helgaas wrote:
> From: Tomasz Nowicki <tn@semihalf.com>
> 
> Currently we use one shared global acpi_pci_root_ops structure to keep
> controller-specific ops. We pass its pointer to acpi_pci_root_create() and
> associate it with a host bridge instance for good.  Such a design implies
> serious drawback. Any potential manipulation on the single system-wide
> acpi_pci_root_ops leads to kernel crash. The structure content is not
> really changing even across multiple host bridges creation; thus it was not
> an issue so far.
> 
> In preparation for adding ECAM quirks mechanism (where controller-specific
> PCI ops may be different for each host bridge) allocate new
> acpi_pci_root_ops and fill in with data for each bridge. Now it is safe to
> have different controller-specific info. As a consequence free
> acpi_pci_root_ops when host bridge is released.
> 
> No functional changes in this patch.
> 
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> ---
>  arch/arm64/kernel/pci.c |   17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)

Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
> index 7909f59..1eb42ba 100644
> --- a/arch/arm64/kernel/pci.c
> +++ b/arch/arm64/kernel/pci.c
> @@ -169,33 +169,36 @@ static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
>  
>  	ri = container_of(ci, struct acpi_pci_generic_root_info, common);
>  	pci_ecam_free(ri->cfg);
> +	kfree(ci->ops);
>  	kfree(ri);
>  }
>  
> -static struct acpi_pci_root_ops acpi_pci_root_ops = {
> -	.release_info = pci_acpi_generic_release_info,
> -};
> -
>  /* Interface called from ACPI code to setup PCI host controller */
>  struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  {
>  	int node = acpi_get_node(root->device->handle);
>  	struct acpi_pci_generic_root_info *ri;
>  	struct pci_bus *bus, *child;
> +	struct acpi_pci_root_ops *root_ops;
>  
>  	ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
>  	if (!ri)
>  		return NULL;
>  
> +	root_ops = kzalloc_node(sizeof(*root_ops), GFP_KERNEL, node);
> +	if (!root_ops)
> +		return NULL;
> +
>  	ri->cfg = pci_acpi_setup_ecam_mapping(root);
>  	if (!ri->cfg) {
>  		kfree(ri);
> +		kfree(root_ops);
>  		return NULL;
>  	}
>  
> -	acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
> -	bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
> -				   ri->cfg);
> +	root_ops->release_info = pci_acpi_generic_release_info;
> +	root_ops->pci_ops = &ri->cfg->ops->pci_ops;
> +	bus = acpi_pci_root_create(root, root_ops, &ri->common, ri->cfg);
>  	if (!bus)
>  		return NULL;
>  
> 

[PATCH v11 06/15] arm64: PCI: Exclude ACPI "consumer" resources from host bridge windows

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

On x86 and ia64, we have treated all ACPI _CRS resources of PNP0A03 host
bridge devices as "producers", i.e., as host bridge windows.  That's partly
because some x86 BIOSes improperly used "consumer" descriptors to describe
windows and partly because Linux didn't have good support for handling
consumer and producer descriptors differently.

One result is that x86 BIOSes describe host bridge "consumer" resources in
the _CRS of a PNP0C02 device, not the PNP0A03 device itself.  On arm64 we
don't have a legacy of firmware that has this consumer/producer confusion,
so we can handle PNP0A03 "consumer" descriptors as host bridge registers
instead of windows.

Exclude non-window ("consumer") resources from the list of host bridge
windows.  This allows the use of "consumer" PNP0A03 descriptors for bridge
register space.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/arm64/kernel/pci.c |   14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 1eb42ba..ecf026d0 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -114,6 +114,19 @@ int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
 	return 0;
 }
 
+static int pci_acpi_root_prepare_resources(struct acpi_pci_root_info *ci)
+{
+	struct resource_entry *entry, *tmp;
+	int status;
+
+	status = acpi_pci_probe_root_resources(ci);
+	resource_list_for_each_entry_safe(entry, tmp, &ci->resources) {
+		if (!(entry->res->flags & IORESOURCE_WINDOW))
+			resource_list_destroy_entry(entry);
+	}
+	return status;
+}
+
 /*
  * Lookup the bus range for the domain in MCFG, and set up config space
  * mapping.
@@ -197,6 +210,7 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
 	}
 
 	root_ops->release_info = pci_acpi_generic_release_info;
+	root_ops->prepare_resources = pci_acpi_root_prepare_resources;
 	root_ops->pci_ops = &ri->cfg->ops->pci_ops;
 	bus = acpi_pci_root_create(root, root_ops, &ri->common, ri->cfg);
 	if (!bus)

[PATCH v11 07/15] PCI/ACPI: Extend pci_mcfg_lookup() to return ECAM config accessors

[Bjorn Helgaas]

From: Tomasz Nowicki <tn@semihalf.com>

pci_mcfg_lookup() is the external interface to the generic MCFG code.
Previously it merely looked up the ECAM base address for a given domain and
bus range.  We want a way to add MCFG quirks, some of which may require
special config accessors and adjustments to the ECAM address range.

Extend pci_mcfg_lookup() so it can return a pointer to a pci_ecam_ops
structure and a struct resource for the ECAM address space.  For now, it
always returns &pci_generic_ecam_ops (the standard accessor) and the
resource described by the MCFG.

No functional changes intended.

[bhelgaas: changelog]
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/arm64/kernel/pci.c  |   17 +++++------------
 drivers/acpi/pci_mcfg.c  |   28 +++++++++++++++++++++++++---
 include/linux/pci-acpi.h |    4 +++-
 3 files changed, 33 insertions(+), 16 deletions(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index ecf026d0..a16fc8e 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -137,25 +137,18 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
 	struct device *dev = &root->device->dev;
 	struct resource *bus_res = &root->secondary;
 	u16 seg = root->segment;
+	struct pci_ecam_ops *ecam_ops;
 	struct resource cfgres;
 	struct acpi_device *adev;
 	struct pci_config_window *cfg;
-	unsigned int bsz;
+	int ret;
 
-	/* Use address from _CBA if present, otherwise lookup MCFG */
-	if (!root->mcfg_addr)
-		root->mcfg_addr = pci_mcfg_lookup(seg, bus_res);
-
-	if (!root->mcfg_addr) {
+	ret = pci_mcfg_lookup(root, &cfgres, &ecam_ops);
+	if (ret) {
 		dev_err(dev, "%04x:%pR ECAM region not found\n", seg, bus_res);
 		return NULL;
 	}
 
-	bsz = 1 << pci_generic_ecam_ops.bus_shift;
-	cfgres.start = root->mcfg_addr + bus_res->start * bsz;
-	cfgres.end = cfgres.start + resource_size(bus_res) * bsz - 1;
-	cfgres.flags = IORESOURCE_MEM;
-
 	adev = acpi_resource_consumer(&cfgres);
 	if (adev)
 		dev_info(dev, "ECAM area %pR reserved by %s\n", &cfgres,
@@ -165,7 +158,7 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
 			 &cfgres);
 
 
-	cfg = pci_ecam_create(dev, &cfgres, bus_res, &pci_generic_ecam_ops);
+	cfg = pci_ecam_create(dev, &cfgres, bus_res, ecam_ops);
 	if (IS_ERR(cfg)) {
 		dev_err(dev, "%04x:%pR error %ld mapping ECAM\n", seg, bus_res,
 			PTR_ERR(cfg));
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index b5b376e..ffcc651 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -22,6 +22,7 @@
 #include <linux/kernel.h>
 #include <linux/pci.h>
 #include <linux/pci-acpi.h>
+#include <linux/pci-ecam.h>
 
 /* Structure to hold entries from the MCFG table */
 struct mcfg_entry {
@@ -35,9 +36,18 @@ struct mcfg_entry {
 /* List to save MCFG entries */
 static LIST_HEAD(pci_mcfg_list);
 
-phys_addr_t pci_mcfg_lookup(u16 seg, struct resource *bus_res)
+int pci_mcfg_lookup(struct acpi_pci_root *root, struct resource *cfgres,
+		    struct pci_ecam_ops **ecam_ops)
 {
+	struct pci_ecam_ops *ops = &pci_generic_ecam_ops;
+	struct resource *bus_res = &root->secondary;
+	u16 seg = root->segment;
 	struct mcfg_entry *e;
+	struct resource res;
+
+	/* Use address from _CBA if present, otherwise lookup MCFG */
+	if (root->mcfg_addr)
+		goto skip_lookup;
 
 	/*
 	 * We expect exact match, unless MCFG entry end bus covers more than
@@ -45,10 +55,22 @@ phys_addr_t pci_mcfg_lookup(u16 seg, struct resource *bus_res)
 	 */
 	list_for_each_entry(e, &pci_mcfg_list, list) {
 		if (e->segment == seg && e->bus_start == bus_res->start &&
-		    e->bus_end >= bus_res->end)
-			return e->addr;
+		    e->bus_end >= bus_res->end) {
+			root->mcfg_addr = e->addr;
+		}
+
 	}
 
+	if (!root->mcfg_addr)
+		return -ENXIO;
+
+skip_lookup:
+	memset(&res, 0, sizeof(res));
+	res.start = root->mcfg_addr + (bus_res->start << 20);
+	res.end = res.start + (resource_size(bus_res) << 20) - 1;
+	res.flags = IORESOURCE_MEM;
+	*cfgres = res;
+	*ecam_ops = ops;
 	return 0;
 }
 
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 7d63a66..7a4e83a 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -24,7 +24,9 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
 }
 extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
 
-extern phys_addr_t pci_mcfg_lookup(u16 domain, struct resource *bus_res);
+struct pci_ecam_ops;
+extern int pci_mcfg_lookup(struct acpi_pci_root *root, struct resource *cfgres,
+			   struct pci_ecam_ops **ecam_ops);
 
 static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
 {

[PATCH v11 08/15] PCI/ACPI: Check for platform-specific MCFG quirks

[Bjorn Helgaas]

From: Tomasz Nowicki <tn@semihalf.com>

The PCIe spec (r3.0, sec 7.2.2) specifies an "Enhanced Configuration Access
Mechanism" (ECAM) for memory-mapped access to configuration space.  ECAM is
required for PCIe systems unless there's a standard firmware interface for
config access.

In the absence of a firmware interface, we use pci_generic_ecam_ops, and on
ACPI systems, we discover the ECAM space via the MCFG table and/or the _CBA
method.

Unfortunately some systems provide MCFG but don't implement ECAM according
to spec, so we need a mechanism for quirks to make those systems work.

Add an MCFG quirk mechanism to override the config accessor functions
and/or the memory-mapped address space.

A quirk is selected if it matches all of the following:

  - OEM ID
  - OEM Table ID
  - OEM Revision
  - PCI segment (from _SEG)
  - PCI bus number range (from _CRS, wildcard allowed)

If the quirk specifies config accessor functions or a memory-mapped address
range, these override the defaults.

[bhelgaas: changelog, reorder quirk matching, fix oem_revision typo per
Duc, add under #ifdef CONFIG_PCI_QUIRKS]
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Dongdong Liu <liudongdong3@huawei.com>
Signed-off-by: Christopher Covington <cov@codeaurora.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/pci_mcfg.c |   92 ++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 86 insertions(+), 6 deletions(-)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index ffcc651..1ef7285 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -33,6 +33,69 @@ struct mcfg_entry {
 	u8			bus_end;
 };
 
+#ifdef CONFIG_PCI_QUIRKS
+struct mcfg_fixup {
+	char oem_id[ACPI_OEM_ID_SIZE + 1];
+	char oem_table_id[ACPI_OEM_TABLE_ID_SIZE + 1];
+	u32 oem_revision;
+	u16 segment;
+	struct resource bus_range;
+	struct pci_ecam_ops *ops;
+	struct resource cfgres;
+};
+
+#define MCFG_BUS_RANGE(start, end)	DEFINE_RES_NAMED((start),	\
+						((end) - (start) + 1),	\
+						NULL, IORESOURCE_BUS)
+#define MCFG_BUS_ANY			MCFG_BUS_RANGE(0x0, 0xff)
+
+static struct mcfg_fixup mcfg_quirks[] = {
+/*	{ OEM_ID, OEM_TABLE_ID, REV, SEGMENT, BUS_RANGE, ops, cfgres }, */
+};
+
+static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
+static char mcfg_oem_table_id[ACPI_OEM_TABLE_ID_SIZE];
+static u32 mcfg_oem_revision;
+
+static int pci_mcfg_quirk_matches(struct mcfg_fixup *f, u16 segment,
+				  struct resource *bus_range)
+{
+	if (!memcmp(f->oem_id, mcfg_oem_id, ACPI_OEM_ID_SIZE) &&
+	    !memcmp(f->oem_table_id, mcfg_oem_table_id,
+	            ACPI_OEM_TABLE_ID_SIZE) &&
+	    f->oem_revision == mcfg_oem_revision &&
+	    f->segment == segment &&
+	    resource_contains(&f->bus_range, bus_range))
+		return 1;
+
+	return 0;
+}
+#endif
+
+static void pci_mcfg_apply_quirks(struct acpi_pci_root *root,
+				  struct resource *cfgres,
+				  struct pci_ecam_ops **ecam_ops)
+{
+#ifdef CONFIG_PCI_QUIRKS
+	u16 segment = root->segment;
+	struct resource *bus_range = &root->secondary;
+	struct mcfg_fixup *f;
+	int i;
+
+	for (i = 0, f = mcfg_quirks; i < ARRAY_SIZE(mcfg_quirks); i++, f++) {
+		if (pci_mcfg_quirk_matches(f, segment, bus_range)) {
+			if (f->cfgres.start)
+				*cfgres = f->cfgres;
+			if (f->ops)
+				*ecam_ops =  f->ops;
+			dev_info(&root->device->dev, "MCFG quirk: ECAM at %pR for %pR with %ps\n",
+				 cfgres, bus_range, *ecam_ops);
+			return;
+		}
+	}
+#endif
+}
+
 /* List to save MCFG entries */
 static LIST_HEAD(pci_mcfg_list);
 
@@ -61,14 +124,24 @@ int pci_mcfg_lookup(struct acpi_pci_root *root, struct resource *cfgres,
 
 	}
 
-	if (!root->mcfg_addr)
-		return -ENXIO;
-
 skip_lookup:
 	memset(&res, 0, sizeof(res));
-	res.start = root->mcfg_addr + (bus_res->start << 20);
-	res.end = res.start + (resource_size(bus_res) << 20) - 1;
-	res.flags = IORESOURCE_MEM;
+	if (root->mcfg_addr) {
+		res.start = root->mcfg_addr + (bus_res->start << 20);
+		res.end = res.start + (resource_size(bus_res) << 20) - 1;
+		res.flags = IORESOURCE_MEM;
+	}
+
+	/*
+	 * Allow quirks to override default ECAM ops and CFG resource
+	 * range.  This may even fabricate a CFG resource range in case
+	 * MCFG does not have it.  Invalid CFG start address means MCFG
+	 * firmware bug or we need another quirk in array.
+	 */
+	pci_mcfg_apply_quirks(root, &res, &ops);
+	if (!res.start)
+		return -ENXIO;
+
 	*cfgres = res;
 	*ecam_ops = ops;
 	return 0;
@@ -101,6 +174,13 @@ static __init int pci_mcfg_parse(struct acpi_table_header *header)
 		list_add(&e->list, &pci_mcfg_list);
 	}
 
+#ifdef CONFIG_PCI_QUIRKS
+	/* Save MCFG IDs and revision for quirks matching */
+	memcpy(mcfg_oem_id, header->oem_id, ACPI_OEM_ID_SIZE);
+	memcpy(mcfg_oem_table_id, header->oem_table_id, ACPI_OEM_TABLE_ID_SIZE);
+	mcfg_oem_revision = header->oem_revision;
+#endif
+
 	pr_info("MCFG table detected, %d entries\n", n);
 	return 0;
 }

[PATCH v11 09/15] PCI/ACPI: Provide acpi_get_rc_resources() for ARM64 platform

[Bjorn Helgaas]

From: Dongdong Liu <liudongdong3@huawei.com>

The acpi_get_rc_resources() is used to get the RC register address that can
not be described in MCFG.  It takes the _HID & segment to look for and
outputs the RC address resource.  Use PNP0C02 devices to describe such RC
address resource.  Use _UID to match segment to tell which root bus the
PNP0C02 resource belongs to.

[bhelgaas: add dev argument, wrap in #ifdef CONFIG_PCI_QUIRKS]
Signed-off-by: Dongdong Liu <liudongdong3@huawei.com>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/pci-acpi.c |   76 ++++++++++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.h      |    5 +++
 2 files changed, 81 insertions(+)

diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
index d966d47..92384de 100644
--- a/drivers/pci/pci-acpi.c
+++ b/drivers/pci/pci-acpi.c
@@ -29,6 +29,82 @@ const u8 pci_acpi_dsm_uuid[] = {
 	0x91, 0x17, 0xea, 0x4d, 0x19, 0xc3, 0x43, 0x4d
 };
 
+#if defined(CONFIG_PCI_QUIRKS) && defined(CONFIG_ARM64)
+static int acpi_get_rc_addr(struct acpi_device *adev, struct resource *res)
+{
+	struct device *dev = &adev->dev;
+	struct resource_entry *entry;
+	struct list_head list;
+	unsigned long flags;
+	int ret;
+
+	INIT_LIST_HEAD(&list);
+	flags = IORESOURCE_MEM;
+	ret = acpi_dev_get_resources(adev, &list,
+				     acpi_dev_filter_resource_type_cb,
+				     (void *) flags);
+	if (ret < 0) {
+		dev_err(dev, "failed to parse _CRS method, error code %d\n",
+			ret);
+		return ret;
+	}
+
+	if (ret == 0) {
+		dev_err(dev, "no IO and memory resources present in _CRS\n");
+		return -EINVAL;
+	}
+
+	entry = list_first_entry(&list, struct resource_entry, node);
+	*res = *entry->res;
+	acpi_dev_free_resource_list(&list);
+	return 0;
+}
+
+static acpi_status acpi_match_rc(acpi_handle handle, u32 lvl, void *context,
+				 void **retval)
+{
+	u16 *segment = context;
+	unsigned long long uid;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(handle, "_UID", NULL, &uid);
+	if (ACPI_FAILURE(status) || uid != *segment)
+		return AE_CTRL_DEPTH;
+
+	*(acpi_handle *)retval = handle;
+	return AE_CTRL_TERMINATE;
+}
+
+int acpi_get_rc_resources(struct device *dev, const char *hid, u16 segment,
+			  struct resource *res)
+{
+	struct acpi_device *adev;
+	acpi_status status;
+	acpi_handle handle;
+	int ret;
+
+	status = acpi_get_devices(hid, acpi_match_rc, &segment, &handle);
+	if (ACPI_FAILURE(status)) {
+		dev_err(dev, "can't find _HID %s device to locate resources\n",
+			hid);
+		return -ENODEV;
+	}
+
+	ret = acpi_bus_get_device(handle, &adev);
+	if (ret)
+		return ret;
+
+	ret = acpi_get_rc_addr(adev, res);
+	if (ret) {
+		dev_err(dev, "can't get resource from %s\n",
+			dev_name(&adev->dev));
+		return ret;
+	}
+
+	return 0;
+}
+#endif
+
 phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle)
 {
 	acpi_status status = AE_NOT_EXIST;
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 4518562..72b128e 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -356,4 +356,9 @@ static inline int pci_dev_specific_reset(struct pci_dev *dev, int probe)
 }
 #endif
 
+#if defined(CONFIG_PCI_QUIRKS) && defined(CONFIG_ARM64)
+int acpi_get_rc_resources(struct device *dev, const char *hid, u16 segment,
+			  struct resource *res);
+#endif
+
 #endif /* DRIVERS_PCI_H */

[PATCH v11 10/15] PCI: Add MCFG quirks for Qualcomm QDF2432 host controller

[Bjorn Helgaas]

From: Christopher Covington <cov@codeaurora.org>

The Qualcomm Technologies QDF2432 SoC does not support accesses smaller
than 32 bits to the PCI configuration space.  Register the appropriate
quirk.

[bhelgaas: add QCOM_ECAM32 macro, ifdef for ACPI and PCI_QUIRKS]
Signed-off-by: Christopher Covington <cov@codeaurora.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/pci_mcfg.c  |   11 +++++++++++
 drivers/pci/ecam.c       |   12 ++++++++++++
 include/linux/pci-ecam.h |    4 ++++
 3 files changed, 27 insertions(+)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index 1ef7285..cee33b0 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -51,6 +51,17 @@ struct mcfg_fixup {
 
 static struct mcfg_fixup mcfg_quirks[] = {
 /*	{ OEM_ID, OEM_TABLE_ID, REV, SEGMENT, BUS_RANGE, ops, cfgres }, */
+
+#define QCOM_ECAM32(seg) \
+	{ "QCOM  ", "QDF2432 ", 1, seg, MCFG_BUS_ANY, &pci_32b_ops }
+	QCOM_ECAM32(0),
+	QCOM_ECAM32(1),
+	QCOM_ECAM32(2),
+	QCOM_ECAM32(3),
+	QCOM_ECAM32(4),
+	QCOM_ECAM32(5),
+	QCOM_ECAM32(6),
+	QCOM_ECAM32(7),
 };
 
 static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index 43ed08d..2fee61b 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -162,3 +162,15 @@ struct pci_ecam_ops pci_generic_ecam_ops = {
 		.write		= pci_generic_config_write,
 	}
 };
+
+#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
+/* ECAM ops for 32-bit access only (non-compliant) */
+struct pci_ecam_ops pci_32b_ops = {
+	.bus_shift	= 20,
+	.pci_ops	= {
+		.map_bus	= pci_ecam_map_bus,
+		.read		= pci_generic_config_read32,
+		.write		= pci_generic_config_write32,
+	}
+};
+#endif
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index 7adad20..739d233 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -59,6 +59,10 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
 /* default ECAM ops */
 extern struct pci_ecam_ops pci_generic_ecam_ops;
 
+#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
+extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
+#endif
+
 #ifdef CONFIG_PCI_HOST_GENERIC
 /* for DT-based PCI controllers that support ECAM */
 int pci_host_common_probe(struct platform_device *pdev,

[PATCH v11 11/15] PCI: Add MCFG quirks for HiSilicon Hip05/06/07 host controllers

[Bjorn Helgaas]

From: Dongdong Liu <liudongdong3@huawei.com>

The PCIe controller in Hip05/Hip06/Hip07 SoCs is not completely
ECAM-compliant.  It is non-ECAM only for the RC bus config space; for any
other bus underneath the root bus it does support ECAM access.

Add specific quirks for PCI config space accessors.  This involves:
1. New initialization call hisi_pcie_init() to obtain RC base
addresses from PNP0C02 at the root of the ACPI namespace (under \_SB).
2. New entry in common quirk array.

[bhelgaas: move to pcie-hisi.c and change Makefile/ifdefs so quirk doesn't
depend on CONFIG_PCI_HISI]
Signed-off-by: Dongdong Liu <liudongdong3@huawei.com>
Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/pci_mcfg.c      |   12 +++++
 drivers/pci/host/Makefile    |    2 -
 drivers/pci/host/pcie-hisi.c |  101 ++++++++++++++++++++++++++++++++++++++++++
 include/linux/pci-ecam.h     |    1 
 4 files changed, 115 insertions(+), 1 deletion(-)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index cee33b0..540515f 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -62,6 +62,18 @@ static struct mcfg_fixup mcfg_quirks[] = {
 	QCOM_ECAM32(5),
 	QCOM_ECAM32(6),
 	QCOM_ECAM32(7),
+
+#define HISI_QUAD_DOM(table_id, seg, ops) \
+	{ "HISI  ", table_id, 0, seg + 0, MCFG_BUS_ANY, ops }, \
+	{ "HISI  ", table_id, 0, seg + 1, MCFG_BUS_ANY, ops }, \
+	{ "HISI  ", table_id, 0, seg + 2, MCFG_BUS_ANY, ops }, \
+	{ "HISI  ", table_id, 0, seg + 3, MCFG_BUS_ANY, ops }
+	HISI_QUAD_DOM("HIP05   ",  0, &hisi_pcie_ops),
+	HISI_QUAD_DOM("HIP06   ",  0, &hisi_pcie_ops),
+	HISI_QUAD_DOM("HIP07   ",  0, &hisi_pcie_ops),
+	HISI_QUAD_DOM("HIP07   ",  4, &hisi_pcie_ops),
+	HISI_QUAD_DOM("HIP07   ",  8, &hisi_pcie_ops),
+	HISI_QUAD_DOM("HIP07   ", 12, &hisi_pcie_ops),
 };
 
 static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index 084cb49..64845f0 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -25,7 +25,7 @@ obj-$(CONFIG_PCIE_IPROC_PLATFORM) += pcie-iproc-platform.o
 obj-$(CONFIG_PCIE_IPROC_BCMA) += pcie-iproc-bcma.o
 obj-$(CONFIG_PCIE_ALTERA) += pcie-altera.o
 obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
-obj-$(CONFIG_PCI_HISI) += pcie-hisi.o
+obj-$(CONFIG_ARM64) += pcie-hisi.o
 obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
 obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
 obj-$(CONFIG_PCI_HOST_THUNDER_PEM) += pci-thunder-pem.o
diff --git a/drivers/pci/host/pcie-hisi.c b/drivers/pci/host/pcie-hisi.c
index 56154c2..9bfa1ab 100644
--- a/drivers/pci/host/pcie-hisi.c
+++ b/drivers/pci/host/pcie-hisi.c
@@ -18,7 +18,106 @@
 #include <linux/of_pci.h>
 #include <linux/platform_device.h>
 #include <linux/of_device.h>
+#include <linux/pci.h>
+#include <linux/pci-acpi.h>
+#include <linux/pci-ecam.h>
 #include <linux/regmap.h>
+#include "../pci.h"
+
+#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
+
+static int hisi_pcie_acpi_rd_conf(struct pci_bus *bus, u32 devfn, int where,
+				  int size, u32 *val)
+{
+	struct pci_config_window *cfg = bus->sysdata;
+	int dev = PCI_SLOT(devfn);
+
+	if (bus->number == cfg->busr.start) {
+		/* access only one slot on each root port */
+		if (dev > 0)
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		else
+			return pci_generic_config_read32(bus, devfn, where,
+							 size, val);
+	}
+
+	return pci_generic_config_read(bus, devfn, where, size, val);
+}
+
+static int hisi_pcie_acpi_wr_conf(struct pci_bus *bus, u32 devfn,
+				  int where, int size, u32 val)
+{
+	struct pci_config_window *cfg = bus->sysdata;
+	int dev = PCI_SLOT(devfn);
+
+	if (bus->number == cfg->busr.start) {
+		/* access only one slot on each root port */
+		if (dev > 0)
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		else
+			return pci_generic_config_write32(bus, devfn, where,
+							  size, val);
+	}
+
+	return pci_generic_config_write(bus, devfn, where, size, val);
+}
+
+static void __iomem *hisi_pcie_map_bus(struct pci_bus *bus, unsigned int devfn,
+				       int where)
+{
+	struct pci_config_window *cfg = bus->sysdata;
+	void __iomem *reg_base = cfg->priv;
+
+	if (bus->number == cfg->busr.start)
+		return reg_base + where;
+	else
+		return pci_ecam_map_bus(bus, devfn, where);
+}
+
+static int hisi_pcie_init(struct pci_config_window *cfg)
+{
+	struct device *dev = cfg->parent;
+	struct acpi_device *adev = to_acpi_device(dev);
+	struct acpi_pci_root *root = acpi_driver_data(adev);
+	struct resource *res;
+	void __iomem *reg_base;
+	int ret;
+
+	/*
+	 * Retrieve RC base and size from a HISI0081 device with _UID
+	 * matching our segment.
+	 */
+	res = devm_kzalloc(dev, sizeof(*res), GFP_KERNEL);
+	if (!res)
+		return -ENOMEM;
+
+	ret = acpi_get_rc_resources(dev, "HISI0081", root->segment, res);
+	if (ret) {
+		dev_err(dev, "can't get rc base address\n");
+		return -ENOMEM;
+	}
+
+	reg_base = devm_ioremap(dev, res->start, resource_size(res));
+	if (!reg_base)
+		return -ENOMEM;
+
+	cfg->priv = reg_base;
+	return 0;
+}
+
+struct pci_ecam_ops hisi_pcie_ops = {
+	.bus_shift    = 20,
+	.init         =  hisi_pcie_init,
+	.pci_ops      = {
+		.map_bus    = hisi_pcie_map_bus,
+		.read       = hisi_pcie_acpi_rd_conf,
+		.write      = hisi_pcie_acpi_wr_conf,
+	}
+};
+
+#endif
+
+#ifdef CONFIG_PCI_HISI
 
 #include "pcie-designware.h"
 
@@ -227,3 +326,5 @@ static struct platform_driver hisi_pcie_driver = {
 	},
 };
 builtin_platform_driver(hisi_pcie_driver);
+
+#endif
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index 739d233..bdacbc8 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -61,6 +61,7 @@ extern struct pci_ecam_ops pci_generic_ecam_ops;
 
 #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
 extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
+extern struct pci_ecam_ops hisi_pcie_ops;	/* HiSilicon */
 #endif
 
 #ifdef CONFIG_PCI_HOST_GENERIC

[PATCH v11 12/15] PCI: thunder-pem: Factor out resource lookup

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

Pull the register resource lookup out of thunder_pem_init() so we can
easily add a corresponding lookup using ACPI.  No functional change
intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/host/pci-thunder-pem.c |   50 +++++++++++++++++++-----------------
 1 file changed, 27 insertions(+), 23 deletions(-)

diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index 6abaf80..c3276ee 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -284,35 +284,16 @@ static int thunder_pem_config_write(struct pci_bus *bus, unsigned int devfn,
 	return pci_generic_config_write(bus, devfn, where, size, val);
 }
 
-static int thunder_pem_init(struct pci_config_window *cfg)
+static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg,
+			    struct resource *res_pem)
 {
-	struct device *dev = cfg->parent;
-	resource_size_t bar4_start;
-	struct resource *res_pem;
 	struct thunder_pem_pci *pem_pci;
-	struct platform_device *pdev;
-
-	/* Only OF support for now */
-	if (!dev->of_node)
-		return -EINVAL;
+	resource_size_t bar4_start;
 
 	pem_pci = devm_kzalloc(dev, sizeof(*pem_pci), GFP_KERNEL);
 	if (!pem_pci)
 		return -ENOMEM;
 
-	pdev = to_platform_device(dev);
-
-	/*
-	 * The second register range is the PEM bridge to the PCIe
-	 * bus.  It has a different config access method than those
-	 * devices behind the bridge.
-	 */
-	res_pem = platform_get_resource(pdev, IORESOURCE_MEM, 1);
-	if (!res_pem) {
-		dev_err(dev, "missing \"reg[1]\"property\n");
-		return -EINVAL;
-	}
-
 	pem_pci->pem_reg_base = devm_ioremap(dev, res_pem->start, 0x10000);
 	if (!pem_pci->pem_reg_base)
 		return -ENOMEM;
@@ -332,9 +313,32 @@ static int thunder_pem_init(struct pci_config_window *cfg)
 	return 0;
 }
 
+static int thunder_pem_platform_init(struct pci_config_window *cfg)
+{
+	struct device *dev = cfg->parent;
+	struct platform_device *pdev = to_platform_device(dev);
+	struct resource *res_pem;
+
+	if (!dev->of_node)
+		return -EINVAL;
+
+	/*
+	 * The second register range is the PEM bridge to the PCIe
+	 * bus.  It has a different config access method than those
+	 * devices behind the bridge.
+	 */
+	res_pem = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+	if (!res_pem) {
+		dev_err(dev, "missing \"reg[1]\"property\n");
+		return -EINVAL;
+	}
+
+	return thunder_pem_init(dev, cfg, res_pem);
+}
+
 static struct pci_ecam_ops pci_thunder_pem_ops = {
 	.bus_shift	= 24,
-	.init		= thunder_pem_init,
+	.init		= thunder_pem_platform_init,
 	.pci_ops	= {
 		.map_bus	= pci_ecam_map_bus,
 		.read		= thunder_pem_config_read,

[PATCH v11 13/15] PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller

[Bjorn Helgaas]

From: Tomasz Nowicki <tn@semihalf.com>

ThunderX PCIe controller to off-chip devices (so-called PEM) is not fully
compliant with ECAM standard. It uses non-standard configuration space
accessors (see thunder_pem_ecam_ops) and custom configuration space
granulation (see bus_shift = 24). In order to access configuration space
and probe PEM as ACPI-based PCI host controller we need to add MCFG quirk
infrastructure. This involves:
1. A new thunder_pem_acpi_init() init function to locate PEM-specific
   register ranges using ACPI.
2. Export PEM thunder_pem_ecam_ops structure so it is visible to MCFG quirk
   code.
3. New quirk entries for each PEM segment. Each contains platform IDs,
   mentioned thunder_pem_ecam_ops and CFG resources.

Quirk is considered for ThunderX silicon pass2.x only which is identified
via MCFG revision 1.

[bhelgaas: adapt to use acpi_get_rc_resources(), update Makefile/ifdefs so
quirk doesn't depend on CONFIG_PCI_HOST_THUNDER_PEM]
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/pci_mcfg.c            |   19 ++++++++++++++++
 drivers/pci/host/Kconfig           |    3 ++
 drivers/pci/host/Makefile          |    2 +-
 drivers/pci/host/pci-thunder-pem.c |   44 ++++++++++++++++++++++++++++++++++++
 include/linux/pci-ecam.h           |    1 +
 5 files changed, 67 insertions(+), 2 deletions(-)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index 540515f..4044b8b 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -74,6 +74,25 @@ static struct mcfg_fixup mcfg_quirks[] = {
 	HISI_QUAD_DOM("HIP07   ",  4, &hisi_pcie_ops),
 	HISI_QUAD_DOM("HIP07   ",  8, &hisi_pcie_ops),
 	HISI_QUAD_DOM("HIP07   ", 12, &hisi_pcie_ops),
+
+#define THUNDER_PEM_RES(addr, seg) \
+	DEFINE_RES_MEM(addr + (seg << 44), 0x39 * SZ_16M)
+#define THUNDER_PEM_QUIRK(rev, seg) \
+	{ "CAVIUM", "THUNDERX", rev, 4 + (10 * seg), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88001f000000UL, seg) },  \
+	{ "CAVIUM", "THUNDERX", rev, 5 + (10 * seg), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x884057000000UL, seg) },  \
+	{ "CAVIUM", "THUNDERX", rev, 6 + (10 * seg), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88808f000000UL, seg) },  \
+	{ "CAVIUM", "THUNDERX", rev, 7 + (10 * seg), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89001f000000UL, seg) },  \
+	{ "CAVIUM", "THUNDERX", rev, 8 + (10 * seg), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x894057000000UL, seg) },  \
+	{ "CAVIUM", "THUNDERX", rev, 9 + (10 * seg), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89808f000000UL, seg) }
+	/* SoC pass2.x */
+	THUNDER_PEM_QUIRK(1, 0UL),
+	THUNDER_PEM_QUIRK(1, 1UL),
 };
 
 static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index d7e7c0a..1239a8e 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig
@@ -240,7 +240,8 @@ config PCIE_QCOM
 
 config PCI_HOST_THUNDER_PEM
 	bool "Cavium Thunder PCIe controller to off-chip devices"
-	depends on OF && ARM64
+	depends on ARM64
+	depends on OF || (ACPI && PCI_QUIRKS)
 	select PCI_HOST_COMMON
 	help
 	  Say Y here if you want PCIe support for CN88XX Cavium Thunder SoCs.
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index 64845f0..97e6bfc 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -28,7 +28,7 @@ obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
 obj-$(CONFIG_ARM64) += pcie-hisi.o
 obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
 obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
-obj-$(CONFIG_PCI_HOST_THUNDER_PEM) += pci-thunder-pem.o
+obj-$(CONFIG_ARM64) += pci-thunder-pem.o
 obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
 obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
 obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index c3276ee..af722eb 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -18,8 +18,12 @@
 #include <linux/init.h>
 #include <linux/of_address.h>
 #include <linux/of_pci.h>
+#include <linux/pci-acpi.h>
 #include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
+#include "../pci.h"
+
+#if defined(CONFIG_PCI_HOST_THUNDER_PEM) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))
 
 #define PEM_CFG_WR 0x28
 #define PEM_CFG_RD 0x30
@@ -313,6 +317,43 @@ static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg,
 	return 0;
 }
 
+#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
+
+static int thunder_pem_acpi_init(struct pci_config_window *cfg)
+{
+	struct device *dev = cfg->parent;
+	struct acpi_device *adev = to_acpi_device(dev);
+	struct acpi_pci_root *root = acpi_driver_data(adev);
+	struct resource *res_pem;
+	int ret;
+
+	res_pem = devm_kzalloc(&adev->dev, sizeof(*res_pem), GFP_KERNEL);
+	if (!res_pem)
+		return -ENOMEM;
+
+	ret = acpi_get_rc_resources(dev, "THRX0002", root->segment, res_pem);
+	if (ret) {
+		dev_err(dev, "can't get rc base address\n");
+		return ret;
+	}
+
+	return thunder_pem_init(dev, cfg, res_pem);
+}
+
+struct pci_ecam_ops thunder_pem_ecam_ops = {
+	.bus_shift	= 24,
+	.init		= thunder_pem_acpi_init,
+	.pci_ops	= {
+		.map_bus	= pci_ecam_map_bus,
+		.read		= thunder_pem_config_read,
+		.write		= thunder_pem_config_write,
+	}
+};
+
+#endif
+
+#ifdef CONFIG_PCI_HOST_THUNDER_PEM
+
 static int thunder_pem_platform_init(struct pci_config_window *cfg)
 {
 	struct device *dev = cfg->parent;
@@ -364,3 +405,6 @@ static struct platform_driver thunder_pem_driver = {
 	.probe = thunder_pem_probe,
 };
 builtin_platform_driver(thunder_pem_driver);
+
+#endif
+#endif
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index bdacbc8..e88d7db 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -62,6 +62,7 @@ extern struct pci_ecam_ops pci_generic_ecam_ops;
 #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
 extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
 extern struct pci_ecam_ops hisi_pcie_ops;	/* HiSilicon */
+extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 2.x */
 #endif
 
 #ifdef CONFIG_PCI_HOST_GENERIC

Re: [PATCH v11 13/15] PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller

[Tomasz Nowicki]

Hi Bjorn,

On 06.12.2016 00:26, Bjorn Helgaas wrote:
> diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
> index d7e7c0a..1239a8e 100644
> --- a/drivers/pci/host/Kconfig
> +++ b/drivers/pci/host/Kconfig
> @@ -240,7 +240,8 @@ config PCIE_QCOM
>
>  config PCI_HOST_THUNDER_PEM
>  	bool "Cavium Thunder PCIe controller to off-chip devices"
> -	depends on OF && ARM64
> +	depends on ARM64
> +	depends on OF || (ACPI && PCI_QUIRKS)
>  	select PCI_HOST_COMMON
>  	help
>  	  Say Y here if you want PCIe support for CN88XX Cavium Thunder SoCs.
> diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> index 64845f0..97e6bfc 100644
> --- a/drivers/pci/host/Makefile
> +++ b/drivers/pci/host/Makefile
> @@ -28,7 +28,7 @@ obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
>  obj-$(CONFIG_ARM64) += pcie-hisi.o
>  obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
>  obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
> -obj-$(CONFIG_PCI_HOST_THUNDER_PEM) += pci-thunder-pem.o
> +obj-$(CONFIG_ARM64) += pci-thunder-pem.o

Since we build pci-thunder-pem.o for ARM64 and ...

>  obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
>  obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
>  obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
> diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
> index c3276ee..af722eb 100644
> --- a/drivers/pci/host/pci-thunder-pem.c
> +++ b/drivers/pci/host/pci-thunder-pem.c
> @@ -18,8 +18,12 @@
>  #include <linux/init.h>
>  #include <linux/of_address.h>
>  #include <linux/of_pci.h>
> +#include <linux/pci-acpi.h>
>  #include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
> +#include "../pci.h"
> +
> +#if defined(CONFIG_PCI_HOST_THUNDER_PEM) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))

have this ^^^ ifdef IMO we can drop Kconfig changes in 
drivers/pci/host/Kconfig. The same for PCI_HOST_THUNDER_ECAM in 
patch[14/15].

Thanks,
Tomasz

Re: [PATCH v11 13/15] PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller

[Bjorn Helgaas]

On Tue, Dec 06, 2016 at 02:07:45PM +0100, Tomasz Nowicki wrote:
> Hi Bjorn,
> 
> On 06.12.2016 00:26, Bjorn Helgaas wrote:
> >diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
> >index d7e7c0a..1239a8e 100644
> >--- a/drivers/pci/host/Kconfig
> >+++ b/drivers/pci/host/Kconfig
> >@@ -240,7 +240,8 @@ config PCIE_QCOM
> >
> > config PCI_HOST_THUNDER_PEM
> > 	bool "Cavium Thunder PCIe controller to off-chip devices"
> >-	depends on OF && ARM64
> >+	depends on ARM64
> >+	depends on OF || (ACPI && PCI_QUIRKS)
> > 	select PCI_HOST_COMMON
> > 	help
> > 	  Say Y here if you want PCIe support for CN88XX Cavium Thunder SoCs.
> >diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> >index 64845f0..97e6bfc 100644
> >--- a/drivers/pci/host/Makefile
> >+++ b/drivers/pci/host/Makefile
> >@@ -28,7 +28,7 @@ obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
> > obj-$(CONFIG_ARM64) += pcie-hisi.o
> > obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
> > obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
> >-obj-$(CONFIG_PCI_HOST_THUNDER_PEM) += pci-thunder-pem.o
> >+obj-$(CONFIG_ARM64) += pci-thunder-pem.o
> 
> Since we build pci-thunder-pem.o for ARM64 and ...
> 
> > obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
> > obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
> > obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
> >diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
> >index c3276ee..af722eb 100644
> >--- a/drivers/pci/host/pci-thunder-pem.c
> >+++ b/drivers/pci/host/pci-thunder-pem.c
> >@@ -18,8 +18,12 @@
> > #include <linux/init.h>
> > #include <linux/of_address.h>
> > #include <linux/of_pci.h>
> >+#include <linux/pci-acpi.h>
> > #include <linux/pci-ecam.h>
> > #include <linux/platform_device.h>
> >+#include "../pci.h"
> >+
> >+#if defined(CONFIG_PCI_HOST_THUNDER_PEM) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))
> 
> have this ^^^ ifdef IMO we can drop Kconfig changes in
> drivers/pci/host/Kconfig. The same for PCI_HOST_THUNDER_ECAM in
> patch[14/15].

I think you're right that we *could* drop the Kconfig changes.  I
didn't actually drop them because I think it might be slightly
confusing that with the previous Kconfig:

  config PCI_HOST_THUNDER_PEM
        bool "Cavium Thunder PCIe controller to off-chip devices"
        depends on OF && ARM64

we still build pci-thunder-pem.o, even when CONFIG_OF isn't defined.

There's no question that this whole thing is confusing no matter what
we do.  That reminds me that I meant to add some comments in the
Makefile.  What do you think about the following:


commit 5d06f9125ec00215fa59b789491fdd51186a4ee1
Author: Bjorn Helgaas <bhelgaas@google.com>
Date:   Tue Dec 6 14:27:59 2016 -0600

    PCI: Explain ARM64 ACPI/MCFG quirk Kconfig and build strategy
    
    Add Makefile comments to explain the Kconfig and build strategy for ARM64
    drivers that work around not-quite-ECAM issues.  No functional change
    intended.
    
    Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index 6cc84b4..bfe3179 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -15,7 +15,6 @@ obj-$(CONFIG_PCIE_SPEAR13XX) += pcie-spear13xx.o
 obj-$(CONFIG_PCI_KEYSTONE) += pci-keystone-dw.o pci-keystone.o
 obj-$(CONFIG_PCIE_XILINX) += pcie-xilinx.o
 obj-$(CONFIG_PCIE_XILINX_NWL) += pcie-xilinx-nwl.o
-obj-$(CONFIG_ARM64) += pci-xgene.o
 obj-$(CONFIG_PCI_XGENE_MSI) += pci-xgene-msi.o
 obj-$(CONFIG_PCI_LAYERSCAPE) += pci-layerscape.o
 obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o
@@ -25,11 +24,23 @@ obj-$(CONFIG_PCIE_IPROC_PLATFORM) += pcie-iproc-platform.o
 obj-$(CONFIG_PCIE_IPROC_BCMA) += pcie-iproc-bcma.o
 obj-$(CONFIG_PCIE_ALTERA) += pcie-altera.o
 obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
-obj-$(CONFIG_ARM64) += pcie-hisi.o
 obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
-obj-$(CONFIG_ARM64) += pci-thunder-ecam.o
-obj-$(CONFIG_ARM64) += pci-thunder-pem.o
 obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
 obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
 obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
 obj-$(CONFIG_VMD) += vmd.o
+
+# The following drivers are for devices that use the generic ACPI
+# pci_root.c driver but don't support standard ECAM config access.
+# They contain MCFG quirks to replace the generic ECAM accessors with
+# device-specific ones that are shared with the DT driver.
+
+# The ACPI driver is generic and should not require driver-specific
+# config options to be enabled, so we always build these drivers on
+# ARM64 and use internal ifdefs to only build the pieces we need
+# depending on whether ACPI, the DT driver, or both are enabled.
+
+obj-$(CONFIG_ARM64) += pcie-hisi.o
+obj-$(CONFIG_ARM64) += pci-thunder-ecam.o
+obj-$(CONFIG_ARM64) += pci-thunder-pem.o
+obj-$(CONFIG_ARM64) += pci-xgene.o

Re: [PATCH v11 13/15] PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller

[Tomasz Nowicki]

On 06.12.2016 21:45, Bjorn Helgaas wrote:
> On Tue, Dec 06, 2016 at 02:07:45PM +0100, Tomasz Nowicki wrote:
>> Hi Bjorn,
>>
>> On 06.12.2016 00:26, Bjorn Helgaas wrote:
>>> diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
>>> index d7e7c0a..1239a8e 100644
>>> --- a/drivers/pci/host/Kconfig
>>> +++ b/drivers/pci/host/Kconfig
>>> @@ -240,7 +240,8 @@ config PCIE_QCOM
>>>
>>> config PCI_HOST_THUNDER_PEM
>>> 	bool "Cavium Thunder PCIe controller to off-chip devices"
>>> -	depends on OF && ARM64
>>> +	depends on ARM64
>>> +	depends on OF || (ACPI && PCI_QUIRKS)
>>> 	select PCI_HOST_COMMON
>>> 	help
>>> 	  Say Y here if you want PCIe support for CN88XX Cavium Thunder SoCs.
>>> diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
>>> index 64845f0..97e6bfc 100644
>>> --- a/drivers/pci/host/Makefile
>>> +++ b/drivers/pci/host/Makefile
>>> @@ -28,7 +28,7 @@ obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
>>> obj-$(CONFIG_ARM64) += pcie-hisi.o
>>> obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
>>> obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
>>> -obj-$(CONFIG_PCI_HOST_THUNDER_PEM) += pci-thunder-pem.o
>>> +obj-$(CONFIG_ARM64) += pci-thunder-pem.o
>>
>> Since we build pci-thunder-pem.o for ARM64 and ...
>>
>>> obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
>>> obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
>>> obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
>>> diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
>>> index c3276ee..af722eb 100644
>>> --- a/drivers/pci/host/pci-thunder-pem.c
>>> +++ b/drivers/pci/host/pci-thunder-pem.c
>>> @@ -18,8 +18,12 @@
>>> #include <linux/init.h>
>>> #include <linux/of_address.h>
>>> #include <linux/of_pci.h>
>>> +#include <linux/pci-acpi.h>
>>> #include <linux/pci-ecam.h>
>>> #include <linux/platform_device.h>
>>> +#include "../pci.h"
>>> +
>>> +#if defined(CONFIG_PCI_HOST_THUNDER_PEM) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))
>>
>> have this ^^^ ifdef IMO we can drop Kconfig changes in
>> drivers/pci/host/Kconfig. The same for PCI_HOST_THUNDER_ECAM in
>> patch[14/15].
>
> I think you're right that we *could* drop the Kconfig changes.  I
> didn't actually drop them because I think it might be slightly
> confusing that with the previous Kconfig:
>
>   config PCI_HOST_THUNDER_PEM
>         bool "Cavium Thunder PCIe controller to off-chip devices"
>         depends on OF && ARM64
>
> we still build pci-thunder-pem.o, even when CONFIG_OF isn't defined.
>
> There's no question that this whole thing is confusing no matter what
> we do.  That reminds me that I meant to add some comments in the
> Makefile.  What do you think about the following:
>
>
> commit 5d06f9125ec00215fa59b789491fdd51186a4ee1
> Author: Bjorn Helgaas <bhelgaas@google.com>
> Date:   Tue Dec 6 14:27:59 2016 -0600
>
>     PCI: Explain ARM64 ACPI/MCFG quirk Kconfig and build strategy
>
>     Add Makefile comments to explain the Kconfig and build strategy for ARM64
>     drivers that work around not-quite-ECAM issues.  No functional change
>     intended.
>
>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>
> diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> index 6cc84b4..bfe3179 100644
> --- a/drivers/pci/host/Makefile
> +++ b/drivers/pci/host/Makefile
> @@ -15,7 +15,6 @@ obj-$(CONFIG_PCIE_SPEAR13XX) += pcie-spear13xx.o
>  obj-$(CONFIG_PCI_KEYSTONE) += pci-keystone-dw.o pci-keystone.o
>  obj-$(CONFIG_PCIE_XILINX) += pcie-xilinx.o
>  obj-$(CONFIG_PCIE_XILINX_NWL) += pcie-xilinx-nwl.o
> -obj-$(CONFIG_ARM64) += pci-xgene.o
>  obj-$(CONFIG_PCI_XGENE_MSI) += pci-xgene-msi.o
>  obj-$(CONFIG_PCI_LAYERSCAPE) += pci-layerscape.o
>  obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o
> @@ -25,11 +24,23 @@ obj-$(CONFIG_PCIE_IPROC_PLATFORM) += pcie-iproc-platform.o
>  obj-$(CONFIG_PCIE_IPROC_BCMA) += pcie-iproc-bcma.o
>  obj-$(CONFIG_PCIE_ALTERA) += pcie-altera.o
>  obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
> -obj-$(CONFIG_ARM64) += pcie-hisi.o
>  obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
> -obj-$(CONFIG_ARM64) += pci-thunder-ecam.o
> -obj-$(CONFIG_ARM64) += pci-thunder-pem.o
>  obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
>  obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
>  obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
>  obj-$(CONFIG_VMD) += vmd.o
> +
> +# The following drivers are for devices that use the generic ACPI
> +# pci_root.c driver but don't support standard ECAM config access.
> +# They contain MCFG quirks to replace the generic ECAM accessors with
> +# device-specific ones that are shared with the DT driver.
> +
> +# The ACPI driver is generic and should not require driver-specific
> +# config options to be enabled, so we always build these drivers on
> +# ARM64 and use internal ifdefs to only build the pieces we need
> +# depending on whether ACPI, the DT driver, or both are enabled.
> +
> +obj-$(CONFIG_ARM64) += pcie-hisi.o
> +obj-$(CONFIG_ARM64) += pci-thunder-ecam.o
> +obj-$(CONFIG_ARM64) += pci-thunder-pem.o
> +obj-$(CONFIG_ARM64) += pci-xgene.o
>

Looks good to me.

Tomasz

[PATCH v11 14/15] PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller

[Bjorn Helgaas]

From: Tomasz Nowicki <tn@semihalf.com>

ThunderX pass1.x requires to emulate the EA headers for on-chip devices
hence it has to use custom pci_thunder_ecam_ops for accessing PCI config
space (pci-thuner-ecam.c). Add new entries to MCFG quirk array where it can
be applied while probing ACPI based PCI host controller.

ThunderX pass1.x is using the same way for accessing off-chip devices
(so-called PEM) as silicon pass-2.x so we need to add PEM quirk entries
too.

Quirk is considered for ThunderX silicon pass1.x only which is identified
via MCFG revision 2.

[bhelgaas: change Makefile/ifdefs so quirk doesn't depend on
CONFIG_PCI_HOST_THUNDER_ECAM]
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/pci_mcfg.c             |   15 +++++++++++++++
 drivers/pci/host/Kconfig            |    3 ++-
 drivers/pci/host/Makefile           |    2 +-
 drivers/pci/host/pci-thunder-ecam.c |    9 ++++++++-
 include/linux/pci-ecam.h            |    3 ++-
 5 files changed, 28 insertions(+), 4 deletions(-)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index 4044b8b..8913c5a 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -93,6 +93,21 @@ static struct mcfg_fixup mcfg_quirks[] = {
 	/* SoC pass2.x */
 	THUNDER_PEM_QUIRK(1, 0UL),
 	THUNDER_PEM_QUIRK(1, 1UL),
+
+#define THUNDER_ECAM_QUIRK(rev, seg)					\
+	{ "CAVIUM", "THUNDERX", rev, seg, MCFG_BUS_ANY,			\
+	&pci_thunder_ecam_ops }
+	/* SoC pass1.x */
+	THUNDER_PEM_QUIRK(2,  0UL),	/* off-chip devices */
+	THUNDER_PEM_QUIRK(2,  1UL),	/* off-chip devices */
+	THUNDER_ECAM_QUIRK(2,  0),
+	THUNDER_ECAM_QUIRK(2,  1),
+	THUNDER_ECAM_QUIRK(2,  2),
+	THUNDER_ECAM_QUIRK(2,  3),
+	THUNDER_ECAM_QUIRK(2, 10),
+	THUNDER_ECAM_QUIRK(2, 11),
+	THUNDER_ECAM_QUIRK(2, 12),
+	THUNDER_ECAM_QUIRK(2, 13),
 };
 
 static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index 1239a8e..c983892 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig
@@ -248,7 +248,8 @@ config PCI_HOST_THUNDER_PEM
 
 config PCI_HOST_THUNDER_ECAM
 	bool "Cavium Thunder ECAM controller to on-chip devices on pass-1.x silicon"
-	depends on OF && ARM64
+	depends on ARM64
+	depends on OF || (ACPI && PCI_QUIRKS)
 	select PCI_HOST_COMMON
 	help
 	  Say Y here if you want ECAM support for CN88XX-Pass-1.x Cavium Thunder SoCs.
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index 97e6bfc..639494a 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -27,7 +27,7 @@ obj-$(CONFIG_PCIE_ALTERA) += pcie-altera.o
 obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
 obj-$(CONFIG_ARM64) += pcie-hisi.o
 obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
-obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
+obj-$(CONFIG_ARM64) += pci-thunder-ecam.o
 obj-$(CONFIG_ARM64) += pci-thunder-pem.o
 obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
 obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c
index d50a3dc..3f54a43 100644
--- a/drivers/pci/host/pci-thunder-ecam.c
+++ b/drivers/pci/host/pci-thunder-ecam.c
@@ -14,6 +14,8 @@
 #include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
 
+#if defined(CONFIG_PCI_HOST_THUNDER_ECAM) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))
+
 static void set_val(u32 v, int where, int size, u32 *val)
 {
 	int shift = (where & 3) * 8;
@@ -346,7 +348,7 @@ static int thunder_ecam_config_write(struct pci_bus *bus, unsigned int devfn,
 	return pci_generic_config_write(bus, devfn, where, size, val);
 }
 
-static struct pci_ecam_ops pci_thunder_ecam_ops = {
+struct pci_ecam_ops pci_thunder_ecam_ops = {
 	.bus_shift	= 20,
 	.pci_ops	= {
 		.map_bus        = pci_ecam_map_bus,
@@ -355,6 +357,8 @@ static struct pci_ecam_ops pci_thunder_ecam_ops = {
 	}
 };
 
+#ifdef CONFIG_PCI_HOST_THUNDER_ECAM
+
 static const struct of_device_id thunder_ecam_of_match[] = {
 	{ .compatible = "cavium,pci-host-thunder-ecam" },
 	{ },
@@ -373,3 +377,6 @@ static struct platform_driver thunder_ecam_driver = {
 	.probe = thunder_ecam_probe,
 };
 builtin_platform_driver(thunder_ecam_driver);
+
+#endif
+#endif
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index e88d7db..00eb8eb 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -62,7 +62,8 @@ extern struct pci_ecam_ops pci_generic_ecam_ops;
 #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
 extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
 extern struct pci_ecam_ops hisi_pcie_ops;	/* HiSilicon */
-extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 2.x */
+extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 1.x & 2.x */
+extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 1.x */
 #endif
 
 #ifdef CONFIG_PCI_HOST_GENERIC

[PATCH v11 15/15] PCI: Add MCFG quirks for X-Gene host controller

[Bjorn Helgaas]

From: Duc Dang <dhdang@apm.com>

PCIe controllers in X-Gene SoCs are not ECAM compliant: software needs to
configure additional controller's register to address device at
bus:dev:function.

Add a quirk to discover controller MMIO register space and configure
controller registers to select and address the target secondary device.

The quirk will only be applied for X-Gene PCIe MCFG table with
OEM revison 1, 2, 3 or 4 (PCIe controller v1 and v2 on X-Gene SoCs).

Signed-off-by: Duc Dang <dhdang@apm.com>
Signed-off-by: Bjorn Helgaas <helgaas@kernel.org>
---
 drivers/acpi/pci_mcfg.c      |   25 ++++++++
 drivers/pci/host/Kconfig     |    4 +
 drivers/pci/host/Makefile    |    2 -
 drivers/pci/host/pci-xgene.c |  126 ++++++++++++++++++++++++++++++++++++++++--
 include/linux/pci-ecam.h     |    2 +
 5 files changed, 149 insertions(+), 10 deletions(-)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index 8913c5a..0b2b980 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -108,6 +108,31 @@ static struct mcfg_fixup mcfg_quirks[] = {
 	THUNDER_ECAM_QUIRK(2, 11),
 	THUNDER_ECAM_QUIRK(2, 12),
 	THUNDER_ECAM_QUIRK(2, 13),
+
+#define XGENE_V1_ECAM_MCFG(rev, seg) \
+	{"APM   ", "XGENE   ", rev, seg, MCFG_BUS_ANY, \
+		&xgene_v1_pcie_ecam_ops }
+#define XGENE_V2_ECAM_MCFG(rev, seg) \
+	{"APM   ", "XGENE   ", rev, seg, MCFG_BUS_ANY, \
+		&xgene_v2_pcie_ecam_ops }
+	/* X-Gene SoC with v1 PCIe controller */
+	XGENE_V1_ECAM_MCFG(1, 0),
+	XGENE_V1_ECAM_MCFG(1, 1),
+	XGENE_V1_ECAM_MCFG(1, 2),
+	XGENE_V1_ECAM_MCFG(1, 3),
+	XGENE_V1_ECAM_MCFG(1, 4),
+	XGENE_V1_ECAM_MCFG(2, 0),
+	XGENE_V1_ECAM_MCFG(2, 1),
+	XGENE_V1_ECAM_MCFG(2, 2),
+	XGENE_V1_ECAM_MCFG(2, 3),
+	XGENE_V1_ECAM_MCFG(2, 4),
+	/* X-Gene SoC with v2.1 PCIe controller */
+	XGENE_V2_ECAM_MCFG(3, 0),
+	XGENE_V2_ECAM_MCFG(3, 1),
+	/* X-Gene SoC with v2.2 PCIe controller */
+	XGENE_V2_ECAM_MCFG(4, 0),
+	XGENE_V2_ECAM_MCFG(4, 1),
+	XGENE_V2_ECAM_MCFG(4, 2),
 };
 
 static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index c983892..1fb5518 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig
@@ -133,8 +133,8 @@ config PCIE_XILINX
 
 config PCI_XGENE
 	bool "X-Gene PCIe controller"
-	depends on ARCH_XGENE
-	depends on OF
+	depends on ARM64
+	depends on OF || (ACPI && PCI_QUIRKS)
 	select PCIEPORTBUS
 	help
 	  Say Y here if you want internal PCI support on APM X-Gene SoC.
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index 639494a..6cc84b4 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -15,7 +15,7 @@ obj-$(CONFIG_PCIE_SPEAR13XX) += pcie-spear13xx.o
 obj-$(CONFIG_PCI_KEYSTONE) += pci-keystone-dw.o pci-keystone.o
 obj-$(CONFIG_PCIE_XILINX) += pcie-xilinx.o
 obj-$(CONFIG_PCIE_XILINX_NWL) += pcie-xilinx-nwl.o
-obj-$(CONFIG_PCI_XGENE) += pci-xgene.o
+obj-$(CONFIG_ARM64) += pci-xgene.o
 obj-$(CONFIG_PCI_XGENE_MSI) += pci-xgene-msi.o
 obj-$(CONFIG_PCI_LAYERSCAPE) += pci-layerscape.o
 obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o
diff --git a/drivers/pci/host/pci-xgene.c b/drivers/pci/host/pci-xgene.c
index 1de23d7..7c3b54b 100644
--- a/drivers/pci/host/pci-xgene.c
+++ b/drivers/pci/host/pci-xgene.c
@@ -27,6 +27,8 @@
 #include <linux/of_irq.h>
 #include <linux/of_pci.h>
 #include <linux/pci.h>
+#include <linux/pci-acpi.h>
+#include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
 #include <linux/slab.h>
 
@@ -64,7 +66,9 @@
 /* PCIe IP version */
 #define XGENE_PCIE_IP_VER_UNKN		0
 #define XGENE_PCIE_IP_VER_1		1
+#define XGENE_PCIE_IP_VER_2		2
 
+#if defined(CONFIG_PCI_XGENE) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))
 struct xgene_pcie_port {
 	struct device_node	*node;
 	struct device		*dev;
@@ -91,13 +95,24 @@ static inline u32 pcie_bar_low_val(u32 addr, u32 flags)
 	return (addr & PCI_BASE_ADDRESS_MEM_MASK) | flags;
 }
 
+static inline struct xgene_pcie_port *pcie_bus_to_port(struct pci_bus *bus)
+{
+	struct pci_config_window *cfg;
+
+	if (acpi_disabled)
+		return (struct xgene_pcie_port *)(bus->sysdata);
+
+	cfg = bus->sysdata;
+	return (struct xgene_pcie_port *)(cfg->priv);
+}
+
 /*
  * When the address bit [17:16] is 2'b01, the Configuration access will be
  * treated as Type 1 and it will be forwarded to external PCIe device.
  */
 static void __iomem *xgene_pcie_get_cfg_base(struct pci_bus *bus)
 {
-	struct xgene_pcie_port *port = bus->sysdata;
+	struct xgene_pcie_port *port = pcie_bus_to_port(bus);
 
 	if (bus->number >= (bus->primary + 1))
 		return port->cfg_base + AXI_EP_CFG_ACCESS;
@@ -111,7 +126,7 @@ static void __iomem *xgene_pcie_get_cfg_base(struct pci_bus *bus)
  */
 static void xgene_pcie_set_rtdid_reg(struct pci_bus *bus, uint devfn)
 {
-	struct xgene_pcie_port *port = bus->sysdata;
+	struct xgene_pcie_port *port = pcie_bus_to_port(bus);
 	unsigned int b, d, f;
 	u32 rtdid_val = 0;
 
@@ -158,7 +173,7 @@ static void __iomem *xgene_pcie_map_bus(struct pci_bus *bus, unsigned int devfn,
 static int xgene_pcie_config_read32(struct pci_bus *bus, unsigned int devfn,
 				    int where, int size, u32 *val)
 {
-	struct xgene_pcie_port *port = bus->sysdata;
+	struct xgene_pcie_port *port = pcie_bus_to_port(bus);
 
 	if (pci_generic_config_read32(bus, devfn, where & ~0x3, 4, val) !=
 	    PCIBIOS_SUCCESSFUL)
@@ -182,13 +197,103 @@ static int xgene_pcie_config_read32(struct pci_bus *bus, unsigned int devfn,
 
 	return PCIBIOS_SUCCESSFUL;
 }
+#endif
 
-static struct pci_ops xgene_pcie_ops = {
-	.map_bus = xgene_pcie_map_bus,
-	.read = xgene_pcie_config_read32,
-	.write = pci_generic_config_write32,
+#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
+static int xgene_get_csr_resource(struct acpi_device *adev,
+				  struct resource *res)
+{
+	struct device *dev = &adev->dev;
+	struct resource_entry *entry;
+	struct list_head list;
+	unsigned long flags;
+	int ret;
+
+	INIT_LIST_HEAD(&list);
+	flags = IORESOURCE_MEM;
+	ret = acpi_dev_get_resources(adev, &list,
+				     acpi_dev_filter_resource_type_cb,
+				     (void *) flags);
+	if (ret < 0) {
+		dev_err(dev, "failed to parse _CRS method, error code %d\n",
+			ret);
+		return ret;
+	}
+
+	if (ret == 0) {
+		dev_err(dev, "no IO and memory resources present in _CRS\n");
+		return -EINVAL;
+	}
+
+	entry = list_first_entry(&list, struct resource_entry, node);
+	*res = *entry->res;
+	acpi_dev_free_resource_list(&list);
+	return 0;
+}
+
+static int xgene_pcie_ecam_init(struct pci_config_window *cfg, u32 ipversion)
+{
+	struct device *dev = cfg->parent;
+	struct acpi_device *adev = to_acpi_device(dev);
+	struct xgene_pcie_port *port;
+	struct resource csr;
+	int ret;
+
+	port = devm_kzalloc(dev, sizeof(*port), GFP_KERNEL);
+	if (!port)
+		return -ENOMEM;
+
+	ret = xgene_get_csr_resource(adev, &csr);
+	if (ret) {
+		dev_err(dev, "can't get CSR resource\n");
+		kfree(port);
+		return ret;
+	}
+	port->csr_base = devm_ioremap_resource(dev, &csr);
+	if (IS_ERR(port->csr_base)) {
+		kfree(port);
+		return -ENOMEM;
+	}
+
+	port->cfg_base = cfg->win;
+	port->version = ipversion;
+
+	cfg->priv = port;
+	return 0;
+}
+
+static int xgene_v1_pcie_ecam_init(struct pci_config_window *cfg)
+{
+	return xgene_pcie_ecam_init(cfg, XGENE_PCIE_IP_VER_1);
+}
+
+struct pci_ecam_ops xgene_v1_pcie_ecam_ops = {
+	.bus_shift      = 16,
+	.init           = xgene_v1_pcie_ecam_init,
+	.pci_ops        = {
+		.map_bus        = xgene_pcie_map_bus,
+		.read           = xgene_pcie_config_read32,
+		.write          = pci_generic_config_write,
+	}
+};
+
+static int xgene_v2_pcie_ecam_init(struct pci_config_window *cfg)
+{
+	return xgene_pcie_ecam_init(cfg, XGENE_PCIE_IP_VER_2);
+}
+
+struct pci_ecam_ops xgene_v2_pcie_ecam_ops = {
+	.bus_shift      = 16,
+	.init           = xgene_v2_pcie_ecam_init,
+	.pci_ops        = {
+		.map_bus        = xgene_pcie_map_bus,
+		.read           = xgene_pcie_config_read32,
+		.write          = pci_generic_config_write,
+	}
 };
+#endif
 
+#if defined(CONFIG_PCI_XGENE)
 static u64 xgene_pcie_set_ib_mask(struct xgene_pcie_port *port, u32 addr,
 				  u32 flags, u64 size)
 {
@@ -521,6 +626,12 @@ static int xgene_pcie_setup(struct xgene_pcie_port *port,
 	return 0;
 }
 
+static struct pci_ops xgene_pcie_ops = {
+	.map_bus = xgene_pcie_map_bus,
+	.read = xgene_pcie_config_read32,
+	.write = pci_generic_config_write32,
+};
+
 static int xgene_pcie_probe_bridge(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
@@ -591,3 +702,4 @@ static struct platform_driver xgene_pcie_driver = {
 	.probe = xgene_pcie_probe_bridge,
 };
 builtin_platform_driver(xgene_pcie_driver);
+#endif
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index 00eb8eb..f0d2b94 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -64,6 +64,8 @@ extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
 extern struct pci_ecam_ops hisi_pcie_ops;	/* HiSilicon */
 extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 1.x & 2.x */
 extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 1.x */
+extern struct pci_ecam_ops xgene_v1_pcie_ecam_ops; /* APM X-Gene PCIe v1 */
+extern struct pci_ecam_ops xgene_v2_pcie_ecam_ops; /* APM X-Gene PCIe v2.x */
 #endif
 
 #ifdef CONFIG_PCI_HOST_GENERIC

Re: [PATCH v11 00/15] PCI: ARM64 ECAM quirks

[Duc Dang]

On Mon, Dec 5, 2016 at 3:25 PM, Bjorn Helgaas <helgaas@kernel.org> wrote:
> Here's another try.  The biggest change is to add the APM X-Gene quirks.
>
> These are on my pci/ecam branch.  Please test and report any issues.
> Please also collect dmesg and /proc/iomem contents and I'll try to
> summarize the current ACPI firmware situation on ARM64 and update the
> writeup with recommendations for future platforms.

Here is dmesg and 'cat /proc/iomem' output on X-Gene Mustang board:


Booting Linux on physical CPU 0x0
Linux version 4.9.0-rc1-17009-g6e475ad (dhdang@dhdang-workstation-01)
(gcc version 4.9.3 20150218 (prerelease) (APM-8.0.10-le) ) #81 SMP
PREEMPT Mon Dec 5 17:45:00 PST 2016
Boot CPU: AArch64 Processor [500f0001]
earlycon: uart8250 at MMIO32 0x000000001c020000 (options '')
bootconsole [uart8250] enabled
efi: Getting EFI parameters from FDT:
efi: EFI v2.40 by X-Gene Mustang Board EFI Oct 17 2016 13:54:05
efi:  ACPI=0x47fa700000  ACPI 2.0=0x47fa700014  SMBIOS
3.0=0x47fa9db000  ESRT=0x47ff006f18
esrt: Reserving ESRT space from 0x00000047ff006f18 to 0x00000047ff006f78.
cma: Reserved 256 MiB at 0x00000040f0000000
ACPI: Early table checksum verification disabled
ACPI: RSDP 0x00000047FA700014 000024 (v02 APM   )
ACPI: XSDT 0x00000047FA6F00E8 000084 (v01 APM    XGENE    00000003
 01000013)
ACPI: FACP 0x00000047FA6C0000 00010C (v05 APM    XGENE    00000003
INTL 20140724)
ACPI: DSDT 0x00000047FA6D0000 005922 (v05 APM    APM88xxx 00000001
INTL 20140724)
ACPI: DBG2 0x00000047FA6E0000 0000AA (v00 APMC0D XGENEDBG 00000000
INTL 20140724)
ACPI: GTDT 0x00000047FA6A0000 000060 (v02 APM    XGENE    00000001
INTL 20140724)
ACPI: MCFG 0x00000047FA690000 00003C (v01 APM    XGENE    00000002
INTL 20140724)
ACPI: SPCR 0x00000047FA680000 000050 (v02 APMC0D XGENESPC 00000000
INTL 20140724)
ACPI: SSDT 0x00000047FA670000 00002D (v02 APM    XGENE    00000001
INTL 20140724)
ACPI: BERT 0x00000047FA660000 000030 (v01 APM    XGENE    00000002
INTL 20140724)
ACPI: HEST 0x00000047FA650000 0002A8 (v01 APM    XGENE    00000002
INTL 20140724)
ACPI: APIC 0x00000047FA640000 0002A4 (v03 APM    XGENE    00000003
 01000013)
ACPI: SSDT 0x00000047FA630000 000063 (v02 REDHAT MACADDRS 00000001
 01000013)
ACPI: SSDT 0x00000047FA620000 000032 (v02 REDHAT UARTCLKS 00000001
 01000013)
ACPI: PCCT 0x00000047FA610000 000300 (v01 APM    XGENE    00000003
 01000013)
ACPI: SPCR: console: uart,mmio,0x1c020000,115200
On node 0 totalpages: 8388608
  DMA zone: 16384 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 1048576 pages, LIFO batch:31
  Normal zone: 114688 pages used for memmap
  Normal zone: 7340032 pages, LIFO batch:31
psci: is not implemented in ACPI.
percpu: Embedded 21 pages/cpu @ffff8007fff16000 s48000 r8192 d29824 u86016
pcpu-alloc: s48000 r8192 d29824 u86016 alloc=21*4096
pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [0] 6 [0] 7
Detected PIPT I-cache on CPU0
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 8257536
Kernel command line: BOOT_IMAGE=/apm-opensource/Image
console=ttyS0,115200 earlycon=uart8250,mmio32,0x1c020000 root=/dev/ram
rw netdev=eth0 debug acpi=force
log_buf_len individual max cpu contribution: 4096 bytes
log_buf_len total cpu_extra contributions: 28672 bytes
log_buf_len min size: 16384 bytes
log_buf_len: 65536 bytes
early log buf free: 12852(78%)
PID hash table entries: 4096 (order: 3, 32768 bytes)
Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
software IO TLB [mem 0x40ebfff000-0x40effff000] (64MB) mapped at
[ffff8000ebfff000-ffff8000efffefff]
Memory: 32615844K/33554432K available (8700K kernel code, 870K rwdata,
3792K rodata, 1024K init, 284K bss, 676444K reserved, 262144K
cma-reserved)
Virtual kernel memory layout:
    modules : 0xffff000000000000 - 0xffff000008000000   (   128 MB)
    vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000   (129022 GB)
      .text : 0xffff000008080000 - 0xffff000008900000   (  8704 KB)
    .rodata : 0xffff000008900000 - 0xffff000008cc0000   (  3840 KB)
      .init : 0xffff000008cc0000 - 0xffff000008dc0000   (  1024 KB)
      .data : 0xffff000008dc0000 - 0xffff000008e99a00   (   871 KB)
       .bss : 0xffff000008e99a00 - 0xffff000008ee0bc0   (   285 KB)
    fixed   : 0xffff7dfffe7fd000 - 0xffff7dfffec00000   (  4108 KB)
    PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000   (    16 MB)
    vmemmap : 0xffff7e0000000000 - 0xffff800000000000   (  2048 GB maximum)
              0xffff7e0000000000 - 0xffff7e0020000000   (   512 MB actual)
    memory  : 0xffff800000000000 - 0xffff800800000000   ( 32768 MB)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
Preemptible hierarchical RCU implementation.
    Build-time adjustment of leaf fanout to 64.
    RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=8.
RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=8
NR_IRQS:64 nr_irqs:64 0
GIC: Using split EOI/Deactivate mode
GICv3: No distributor detected at @ffff000008010000, giving up
arm_arch_timer: Architected cp15 timer(s) running at 50.00MHz (phys).
clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles:
0xb8812736b, max_idle_ns: 440795202655 ns
sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
Console: colour dummy device 80x25
Calibrating delay loop (skipped), value calculated using timer
frequency.. 100.00 BogoMIPS (lpj=200000)
pid_max: default: 32768 minimum: 301
ACPI: Core revision 20160831
ACPI Error: Method parse/execution failed [\_SB.ET00._STA] (Node
ffff8007fa9fcdc0), AE_CTRL_PARSE_CONTINUE (20160831/psparse-543)
ACPI Error: Invalid zero thread count in method (20160831/dsmethod-796)
ACPI Error: Invalid OwnerId: 0x00 (20160831/utownerid-186)
ACPI Error: Method parse/execution failed [\_SB.ET01._STA] (Node
ffff8007fa9fe078), AE_CTRL_PARSE_CONTINUE (20160831/psparse-543)
ACPI Error: Invalid zero thread count in method (20160831/dsmethod-796)
ACPI Error: Invalid OwnerId: 0x00 (20160831/utownerid-186)
ACPI: 4 ACPI AML tables successfully acquired and loaded

Security Framework initialized
Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)
Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)
ASID allocator initialised with 65536 entries
Remapping and enabling EFI services.
  EFI remap 0x0000000010510000 => 0000000020000000
  EFI remap 0x0000000010548000 => 0000000020018000
  EFI remap 0x0000000017000000 => 0000000020020000
  EFI remap 0x000000001c025000 => 0000000020035000
  EFI remap 0x00000047fa5a0000 => 0000000020040000
  EFI remap 0x00000047fa5b0000 => 0000000020050000
  EFI remap 0x00000047fa5c0000 => 0000000020060000
  EFI remap 0x00000047fa710000 => 0000000020070000
  EFI remap 0x00000047fa730000 => 0000000020090000
  EFI remap 0x00000047fa790000 => 00000000200f0000
  EFI remap 0x00000047fa7a0000 => 0000000020100000
  EFI remap 0x00000047fa9a0000 => 0000000020300000
  EFI remap 0x00000047fa9b0000 => 0000000020310000
  EFI remap 0x00000047ff9a0000 => 0000000020330000
  EFI remap 0x00000047ff9c0000 => 0000000020340000
Detected PIPT I-cache on CPU1
CPU1: Booted secondary processor [500f0001]
Detected PIPT I-cache on CPU2
CPU2: Booted secondary processor [500f0001]
Detected PIPT I-cache on CPU3
CPU3: Booted secondary processor [500f0001]
Detected PIPT I-cache on CPU4
CPU4: Booted secondary processor [500f0001]
Detected PIPT I-cache on CPU5
CPU5: Booted secondary processor [500f0001]
Detected PIPT I-cache on CPU6
CPU6: Booted secondary processor [500f0001]
Detected PIPT I-cache on CPU7
CPU7: Booted secondary processor [500f0001]
Brought up 8 CPUs
SMP: Total of 8 processors activated.
CPU features: detected feature: 32-bit EL0 Support
CPU: All CPU(s) started at EL2
devtmpfs: initialized
SMBIOS 3.0.0 present.
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff,
max_idle_ns: 7645041785100000 ns
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
cpuidle: using governor menu
vdso: 2 pages (1 code @ ffff000008907000, 1 data @ ffff000008dc4000)
hw-breakpoint: found 4 breakpoint and 4 watchpoint registers.
DMA: preallocated 256 KiB pool for atomic allocations
ACPI: bus type PCI registered
Serial: AMBA PL011 UART driver
HugeTLB registered 2 MB page size, pre-allocated 0 pages
ACPI: Added _OSI(Module Device)
ACPI: Added _OSI(Processor Device)
ACPI: Added _OSI(3.0 _SCP Extensions)
ACPI: Added _OSI(Processor Aggregator Device)
ACPI: Interpreter enabled
ACPI: Using GIC for interrupt routing
ACPI: MCFG table detected, 1 entries
ACPI: Power Resource [SCVR] (on)
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability]
acpi PNP0A08:00: MCFG quirk: ECAM at [mem 0xe0d0000000-0xe0dfffffff]
for [bus 00-ff] with xgene_v1_pcie_ecam_ops
acpi PNP0A08:00: [Firmware Bug]: ECAM area [mem
0xe0d0000000-0xe0dfffffff] not reserved in ACPI namespace
acpi PNP0A08:00: ECAM at [mem 0xe0d0000000-0xe0dfffffff] for [bus 00-ff]
Remapped I/O 0x000000e010000000 to [io  0x0000-0xffff window]
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x0000-0xffff window] (bus
address [0x10000000-0x1000ffff])
pci_bus 0000:00: root bus resource [mem 0xe040000000-0xe07fffffff
window] (bus address [0x40000000-0x7fffffff])
pci_bus 0000:00: root bus resource [mem 0xf000000000-0xffffffffff window]
pci_bus 0000:00: root bus resource [bus 00-ff]
pci 0000:00:00.0: [10e8:e004] type 01 class 0x060400
pci 0000:00:00.0: supports D1 D2
pci 0000:01:00.0: [15b3:1003] type 00 class 0x020000
pci 0000:01:00.0: reg 0x10: [mem 0xe040000000-0xe0400fffff 64bit]
pci 0000:01:00.0: reg 0x18: [mem 0xe042000000-0xe043ffffff 64bit pref]
pci 0000:01:00.0: reg 0x30: [mem 0xfff00000-0xffffffff pref]
pci_bus 0000:00: on NUMA node 0
pci 0000:00:00.0: BAR 15: assigned [mem 0xf000000000-0xf001ffffff 64bit pref]
pci 0000:00:00.0: BAR 14: assigned [mem 0xe040000000-0xe0401fffff]
pci 0000:01:00.0: BAR 2: assigned [mem 0xf000000000-0xf001ffffff 64bit pref]
pci 0000:01:00.0: BAR 0: assigned [mem 0xe040000000-0xe0400fffff 64bit]
pci 0000:01:00.0: BAR 6: assigned [mem 0xe040100000-0xe0401fffff pref]
pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0:   bridge window [mem 0xe040000000-0xe0401fffff]
pci 0000:00:00.0:   bridge window [mem 0xf000000000-0xf001ffffff 64bit pref]
vgaarb: loaded
SCSI subsystem initialized
libata version 3.00 loaded.
ACPI: bus type USB registered
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti
<giometti@linux.it>
PTP clock support registered
Registered efivars operations
Advanced Linux Sound Architecture Driver Initialized.
clocksource: Switched to clocksource arch_sys_counter
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
pnp: PnP ACPI init
pnp: PnP ACPI: found 0 devices
NET: Registered protocol family 2
TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
UDP hash table entries: 16384 (order: 7, 524288 bytes)
UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
PCI: CLS 0 bytes, default 128
Unpacking initramfs...
Freeing initrd memory: 14676K (ffff8007f8767000 - ffff8007f95bc000)
kvm [1]: 8-bit VMID
kvm [1]: IDMAP page: 4000af5000
kvm [1]: HYP VA range: 800000000000:ffffffffffff
kvm [1]: Hyp mode initialized successfully
kvm [1]: vgic-v2@780cf000
kvm [1]: vgic interrupt IRQ1
kvm [1]: virtual timer IRQ4
futex hash table entries: 2048 (order: 6, 262144 bytes)
audit: initializing netlink subsys (disabled)
audit: type=2000 audit(4.120:1): initialized
workingset: timestamp_bits=46 max_order=23 bucket_order=0
squashfs: version 4.0 (2009/01/31) Phillip Lougher
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
nfs4filelayout_init: NFSv4 File Layout Driver Registering...
9p: Installing v9fs 9p2000 file system support
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
io scheduler noop registered
io scheduler cfq registered (default)
libphy: mdio_driver_register: phy-bcm-ns2-pci
xgene-gpio APMC0D14:00: X-Gene GPIO driver registered.
aer 0000:00:00.0:pcie002: service driver aer loaded
pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt
pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
pcie_pme 0000:00:00.0:pcie001: service driver pcie_pme loaded
input: Power Button as /devices/LNXSYSTM:00/PNP0C0C:00/input/input0
ACPI: Power Button [PWRB]
xenfs: not registering filesystem on non-xen platform
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
console [ttyS0] disabled
APMC0D08:00: ttyS0 at MMIO 0x1c020000 (irq = 22, base_baud = 3125000)
is a U6_16550A
console [ttyS0] enabled
bootconsole [uart8250] disabled
APMC0D08:01: ttyS1 at MMIO 0x1c021000 (irq = 23, base_baud = 3125000)
is a 16550A
SuperH (H)SCI(F) driver initialized
msm_serial: driver initialized
Failed to find cpu0 device node
Unable to detect cache hierarchy from DT for CPU 0
loop: module loaded
hisi_sas: driver version v1.6
xgene-ahci APMC0D0D:00: skip clock and PHY initialization
xgene-ahci APMC0D0D:00: controller can't do NCQ, turning off CAP_NCQ
xgene-ahci APMC0D0D:00: AHCI 0001.0300 32 slots 2 ports 6 Gbps 0x3
impl platform mode
xgene-ahci APMC0D0D:00: flags: 64bit sntf pm only pmp fbs pio slum part ccc
xgene-ahci APMC0D0D:00: port 0 is not capable of FBS
xgene-ahci APMC0D0D:00: port 1 is not capable of FBS
scsi host0: xgene-ahci
scsi host1: xgene-ahci
ata1: SATA max UDMA/133 mmio [mem 0x1a400000-0x1a400fff] port 0x100 irq 27
ata2: SATA max UDMA/133 mmio [mem 0x1a400000-0x1a400fff] port 0x180 irq 27
xgene-ahci APMC0D0D:01: skip clock and PHY initialization
xgene-ahci APMC0D0D:01: controller can't do NCQ, turning off CAP_NCQ
xgene-ahci APMC0D0D:01: AHCI 0001.0300 32 slots 2 ports 6 Gbps 0x3
impl platform mode
xgene-ahci APMC0D0D:01: flags: 64bit sntf pm only pmp fbs pio slum part ccc
xgene-ahci APMC0D0D:01: port 0 is not capable of FBS
xgene-ahci APMC0D0D:01: port 1 is not capable of FBS
scsi host2: xgene-ahci
scsi host3: xgene-ahci
ata3: SATA max UDMA/133 mmio [mem 0x1a800000-0x1a800fff] port 0x100 irq 28
ata4: SATA max UDMA/133 mmio [mem 0x1a800000-0x1a800fff] port 0x180 irq 28
libphy: APM X-Gene MDIO bus: probed
libphy: Fixed MDIO Bus: probed
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
xgene-enet APMC0D05:00: clocks have been setup already
xgene-enet APMC0D30:00: clocks have been setup already
ata2: SATA link down (SStatus 0 SControl 4300)
ata1: SATA link down (SStatus 0 SControl 4300)
xgene-enet APMC0D30:01: clocks have been setup already
xgene-enet APMC0D31:00: clocks have been setup already
e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
igb: Copyright (c) 2007-2014 Intel Corporation.
igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
ata3: SATA link down (SStatus 0 SControl 4300)
igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
sky2: driver version 1.30
mlx4_core: Mellanox ConnectX core driver v2.2-1 (Feb, 2014)
mlx4_core: Initializing 0000:01:00.0
ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 4300)
ata4.00: ATA-8: SDLFOCAM-800G-1HA1, ZZ37RE92, max UDMA/133
ata4.00: 1562824368 sectors, multi 0: LBA48 NCQ (depth 0/32)
ata4.00: configured for UDMA/133
scsi 3:0:0:0: Direct-Access     ATA      SDLFOCAM-800G-1H RE92 PQ: 0 ANSI: 5
sd 3:0:0:0: [sda] 1562824368 512-byte logical blocks: (800 GB/745 GiB)
sd 3:0:0:0: [sda] 4096-byte physical blocks
sd 3:0:0:0: [sda] Write Protect is off
sd 3:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 3:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't
support DPO or FUA
 sda: sda1 sda2 sda3
sd 3:0:0:0: [sda] Attached SCSI disk
mlx4_core 0000:01:00.0: PCIe link speed is 8.0GT/s, device supports 8.0GT/s
mlx4_en: Mellanox ConnectX HCA Ethernet driver v2.2-1 (Feb 2014)
mlx4_en 0000:01:00.0: Activating port:1
mlx4_en: 0000:01:00.0: Port 1: Using 64 TX rings
mlx4_en: 0000:01:00.0: Port 1: Using 4 RX rings
mlx4_en: 0000:01:00.0: Port 1:   frag:0 - size:1522 prefix:0 stride:1536
mlx4_en: 0000:01:00.0: Port 1: Initializing port
mlx4_en 0000:01:00.0: registered PHC clock
mlx4_en 0000:01:00.0: Activating port:2
mlx4_en: 0000:01:00.0: Port 2: Using 64 TX rings
mlx4_en: 0000:01:00.0: Port 2: Using 4 RX rings
mlx4_en: 0000:01:00.0: Port 2:   frag:0 - size:1522 prefix:0 stride:1536
mlx4_en: 0000:01:00.0: Port 2: Initializing port
VFIO - User Level meta-driver version: 0.3
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
ehci-platform: EHCI generic platform driver
ehci-exynos: EHCI EXYNOS driver
ehci-msm: Qualcomm On-Chip EHCI Host Controller
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-pci: OHCI PCI platform driver
ohci-platform: OHCI generic platform driver
ohci-exynos: OHCI EXYNOS driver
xhci-hcd: probe of xhci-hcd.0.auto failed with error -5
xhci-hcd: probe of xhci-hcd.1.auto failed with error -5
usbcore: registered new interface driver usb-storage
mousedev: PS/2 mouse device common for all mice
rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
i2c /dev entries driver
sdhci: Secure Digital Host Controller Interface driver
sdhci: Copyright(c) Pierre Ossman
Synopsys Designware Multimedia Card Interface Driver
sdhci-pltfm: SDHCI platform and OF driver helper
ledtrig-cpu: registered to indicate activity on CPUs
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
NET: Registered protocol family 17
9pnet: Installing 9P2000 support
Key type dns_resolver registered
registered taskstats version 1
rtc-efi rtc-efi: setting system clock to 2016-12-06 01:54:38 UTC (1480989278)
ALSA device list:
  No soundcards found.
Freeing unused kernel memory: 1024K (ffff800000ec0000 - ffff800000fc0000)
udevd[1485]: starting version 182
random: fast init done
[root@(none) ~]#
[root@(none) ~]#
[root@(none) ~]#
[root@(none) ~]# cat /proc/iomem
10000000-103fffff : APMC0D05:00
10520000-10523fff : APMC0D18:00
10524000-10527fff : APMC0D17:00
10540000-10547fff : APMC0D01:00
1054a000-1054a0ff : APMC0D43:00
  1054a000-1054a01b : APMC0D41:00
17001000-170013ff : APMC0D15:00
1701c000-1701cfff : APMC0D14:00
17020000-1702d0ff : APMC0D05:00
  17020000-1702d0ff : APMC0D65:00
    17020000-1702d0ff : APMC0D3E:00
      17020000-1702d0ff : APMC0D65:00
17030000-1703ffff : APMC0D05:00
18000000-183fffff : APMC0D31:00
  18000000-183fffff : APMC0D6A:00
19000000-19007fff : 808622B7:00
1900c100-190fffff : 808622B7:00
  1900c100-190fffff : 808622B7:00
19800000-19807fff : 808622B7:01
1980c100-198fffff : 808622B7:01
  1980c100-198fffff : 808622B7:01
1a400000-1a400fff : APMC0D0D:00
  1a400000-1a400fff : APMC0D0D:00
1a800000-1a800fff : APMC0D0D:01
  1a800000-1a800fff : APMC0D0D:01
1b000000-1b3fffff : APMC0D43:00
  1b000000-1b007fff : APMC0D30:01
    1b000000-1b001fff : APMC0D30:00
  1b00a000-1b00bfff : APMC0D41:00
1c000000-1c0000ff : APMC0D0C:00
1c020000-1c0200ff : APMC0D08:00
  1c020000-1c02001f : serial
1c021000-1c0210ff : APMC0D08:01
  1c021000-1c02101f : serial
1c024000-1c024fff : APMC0D07:00
  1c024000-1c024fff : APMC0D07:00
1f200000-1f20ffff : APMC0D41:00
  1f200000-1f20ffff : APMC0D43:00
    1f200000-1f20c2ff : APMC0D30:01
      1f200000-1f20c2ff : APMC0D30:00
1f210000-1f21d0ff : APMC0D30:00
  1f210030-1f21d0ff : APMC0D30:01
1f220000-1f220fff : APMC0D0D:00
  1f220000-1f220fff : APMC0D0D:00
1f227000-1f227fff : APMC0D0D:00
  1f227000-1f227fff : APMC0D0D:00
1f22d000-1f22dfff : APMC0D0D:00
  1f22d000-1f22dfff : APMC0D0D:00
1f22e000-1f22efff : APMC0D0D:00
  1f22e000-1f22efff : APMC0D0D:00
1f230000-1f230fff : APMC0D0D:01
  1f230000-1f230fff : APMC0D0D:01
1f23d000-1f23dfff : APMC0D0D:01
  1f23d000-1f23dfff : APMC0D0D:01
1f23e000-1f23efff : APMC0D0D:01
  1f23e000-1f23efff : APMC0D0D:01
1f250000-1f25ffff : APMC0D41:00
1f270000-1f27ffff : APMC0D43:00
1f280000-1f28ffff : 808622B7:00
1f290000-1f29ffff : 808622B7:01
1f2a0000-1f2a0fff : APMC0D0C:00
1f2b0000-1f2bffff : PNP0A08:00
1f600000-1f60ffff : APMC0D31:00
  1f600000-1f60ffff : APMC0D6A:00
1f610000-1f61ffff : APMC0D31:00
78810000-78810fff : APMC0D5C:00
79000000-798fffff : APMC0D0E:00
7e200000-7e200fff : APMC0D5C:00
7e610000-7e610fff : APMC0D5D:00
7e700000-7e700fff : APMC0D5C:00
7e710000-7e710fff : APMC0D5F:00
7e720000-7e720fff : APMC0D5C:00
7e730000-7e730fff : APMC0D5F:01
7e810000-7e810fff : APMC0D60:00
7e850000-7e850fff : APMC0D60:01
7e890000-7e890fff : APMC0D60:02
7e8d0000-7e8d0fff : APMC0D60:03
7e940000-7e940fff : APMC0D5E:00
4000000000-40001fffff : reserved
4000200000-47fa59ffff : System RAM
  4000280000-4000ebffff : Kernel code
  4000fc0000-40010e6fff : Kernel data
47fa5a0000-47fa5cffff : reserved
47fa5d0000-47fa5ddfff : System RAM
47fa5de000-47fa9cffff : reserved
47fa9d0000-47fa9d9fff : System RAM
47fa9da000-47fa9dbfff : reserved
47fa9dc000-47ff99ffff : System RAM
47ff9a0000-47ff9affff : reserved
47ff9b0000-47ff9bffff : System RAM
47ff9c0000-47ff9effff : reserved
47ff9f0000-47ffffffff : System RAM
e040000000-e07fffffff : PCI Bus 0000:00
  e040000000-e0401fffff : PCI Bus 0000:01
    e040000000-e0400fffff : 0000:01:00.0
      e040000000-e0400fffff : mlx4_core
    e040100000-e0401fffff : 0000:01:00.0
e0d0000000-e0dfffffff : PCI ECAM
f000000000-ffffffffff : PCI Bus 0000:00
  f000000000-f001ffffff : PCI Bus 0000:01
    f000000000-f001ffffff : 0000:01:00.0
      f000000000-f001ffffff : mlx4_core
[root@(none) ~]#

Regards,
Duc Dang.
>
> Changes from v10 to v11:
>
> - Discard "consumer" resources from PNP0A03 host bridge windows (arm64
>   only).  This allows using consumer PNP0A03 descriptors (rather than the
>   PNP0C02 workaround used on x86) for bridge register space.
>
> - Add APM X-Gene MCFG quirks.
>
> - If ACPI and quirks are enabled, always build ThunderX, X-Gene, and
>   HiSilicon drivers on ARM64.  Add internal ifdefs so we only compile the
>   parts necessary for ACPI (the ECAM init and accessors).
>
> - s/node/seg/ in ThunderX MCFG quirk macros.
>
> - Move ECAM check for ACPI device reservation from pci_ecam_create() to
>   that arm64-specific pci_acpi_setup_ecam_mapping().
>
> - Add local "struct device *dev" pointers for brevity.
>
> - Miscellaneous build fixes.
>
> ---
>
> Bjorn Helgaas (6):
>       ACPI: Add acpi_resource_consumer() to find device that claims a resource
>       x86/PCI: Use acpi_resource_consumer() to search ACPI namespace for MMCFG
>       arm64: PCI: Add local struct device pointers
>       arm64: PCI: Search ACPI namespace to ensure ECAM space is reserved
>       arm64: PCI: Exclude ACPI "consumer" resources from host bridge windows
>       PCI: thunder-pem: Factor out resource lookup
>
> Christopher Covington (1):
>       PCI: Add MCFG quirks for Qualcomm QDF2432 host controller
>
> Dongdong Liu (2):
>       PCI/ACPI: Provide acpi_get_rc_resources() for ARM64 platform
>       PCI: Add MCFG quirks for HiSilicon Hip05/06/07 host controllers
>
> Duc Dang (1):
>       PCI: Add MCFG quirks for X-Gene host controller
>
> Tomasz Nowicki (5):
>       arm64: PCI: Manage controller-specific data on per-controller basis
>       PCI/ACPI: Extend pci_mcfg_lookup() to return ECAM config accessors
>       PCI/ACPI: Check for platform-specific MCFG quirks
>       PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller
>       PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller
>
>
>  arch/arm64/kernel/pci.c             |   68 ++++++++-----
>  arch/x86/pci/mmconfig-shared.c      |   69 ++-----------
>  drivers/acpi/pci_mcfg.c             |  190 ++++++++++++++++++++++++++++++++++-
>  drivers/acpi/resource.c             |   57 +++++++++++
>  drivers/pci/ecam.c                  |   12 ++
>  drivers/pci/host/Kconfig            |   10 +-
>  drivers/pci/host/Makefile           |    8 +
>  drivers/pci/host/pci-thunder-ecam.c |    9 +-
>  drivers/pci/host/pci-thunder-pem.c  |   94 +++++++++++++----
>  drivers/pci/host/pci-xgene.c        |  126 ++++++++++++++++++++++-
>  drivers/pci/host/pcie-hisi.c        |  101 +++++++++++++++++++
>  drivers/pci/pci-acpi.c              |   76 ++++++++++++++
>  drivers/pci/pci.h                   |    5 +
>  include/linux/acpi.h                |    7 +
>  include/linux/pci-acpi.h            |    4 +
>  include/linux/pci-ecam.h            |    9 ++
>  16 files changed, 718 insertions(+), 127 deletions(-)

Re: [PATCH v11 00/15] PCI: ARM64 ECAM quirks

[Dongdong Liu]

在 2016/12/6 7:25, Bjorn Helgaas 写道:
> Here's another try.  The biggest change is to add the APM X-Gene quirks.
>
> These are on my pci/ecam branch.  Please test and report any issues.
> Please also collect dmesg and /proc/iomem contents and I'll try to
> summarize the current ACPI firmware situation on ARM64 and update the
> writeup with recommendations for future platforms.

The dmesg and /proc/iomem information are as below on HiSilicon D03 board.
root@(none)$ dmesg
[    0.000000] Booting Linux on physical CPU 0x10000
[    0.000000] Linux version 4.9.0-rc1-g7e48b6b (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #261 SMP PREEMPT Tue Dec 6 11:32:41 CST 2016
[    0.000000] Boot CPU: AArch64 Processor [411fd071]
[    0.000000] earlycon: hisilpcuart0 at MMIO 0x00000000a01b0000 (options '0,0x2f8')
[    0.000000] bootconsole [hisilpcuart0] enabled
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: EFI v2.60 by EDK II
[    0.000000] efi:  SMBIOS=0x3f110000  SMBIOS 3.0=0x39ce0000  ACPI=0x39db0000  ACPI 2.0=0x39db0014  MEMATTR=0x3c943018
[    0.000000] cma: Reserved 16 MiB at 0x000000003e000000
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x0000000039DB0014 000024 (v02 HISI  )
[    0.000000] ACPI: XSDT 0x0000000039DA00E8 00004C (v01 HISI   HIP06    00000000      01000013)
[    0.000000] ACPI: FACP 0x0000000039CA0000 00010C (v05 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: DSDT 0x0000000039C60000 003B02 (v01 HISI   HIP06    00000000 INTL 20150619)
[    0.000000] ACPI: MCFG 0x0000000039CC0000 00005C (v01 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: GTDT 0x0000000039C90000 000098 (v02 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: APIC 0x0000000039C80000 000518 (v01 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: IORT 0x0000000039C70000 0002E8 (v00 HISI   HIP06    00000000 INTL 20150619)
[    0.000000] ACPI: NUMA: Failed to initialise from firmware
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000027ffffffff]
[    0.000000] NUMA: Adding memblock [0x0 - 0x257ff] on node 0
[    0.000000] NUMA: Adding memblock [0x25800 - 0x319fffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31a00000 - 0x31a6ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31a70000 - 0x31b5ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b60000 - 0x31b6ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b70000 - 0x31b7dfff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b7e000 - 0x31b81fff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b82000 - 0x39c0ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x39c10000 - 0x3a11ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3a120000 - 0x3f10ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3f110000 - 0x3f13ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3f140000 - 0x3fbfffff] on node 0
[    0.000000] NUMA: Adding memblock [0x2040000000 - 0x27ffffffff] on node 0
[    0.000000] NUMA: Initmem setup node 0 [mem 0x00000000-0x27ffffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x27fffef500-0x27ffff0fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x00000027ffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x0000000000024fff]
[    0.000000]   node   0: [mem 0x0000000000026000-0x00000000319fffff]
[    0.000000]   node   0: [mem 0x0000000031a00000-0x0000000031a6ffff]
[    0.000000]   node   0: [mem 0x0000000031a70000-0x0000000031b5ffff]
[    0.000000]   node   0: [mem 0x0000000031b60000-0x0000000031b6ffff]
[    0.000000]   node   0: [mem 0x0000000031b70000-0x0000000031b7dfff]
[    0.000000]   node   0: [mem 0x0000000031b7e000-0x0000000031b81fff]
[    0.000000]   node   0: [mem 0x0000000031b82000-0x0000000039c0ffff]
[    0.000000]   node   0: [mem 0x0000000039c10000-0x000000003a11ffff]
[    0.000000]   node   0: [mem 0x000000003a120000-0x000000003f10ffff]
[    0.000000]   node   0: [mem 0x000000003f110000-0x000000003f13ffff]
[    0.000000]   node   0: [mem 0x000000003f140000-0x000000003fbfffff]
[    0.000000]   node   0: [mem 0x0000002040000000-0x00000027ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000027ffffffff]
[    0.000000] On node 0 totalpages: 8387583
[    0.000000]   DMA zone: 4080 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 261119 pages, LIFO batch:31
[    0.000000]   Normal zone: 126976 pages used for memmap
[    0.000000]   Normal zone: 8126464 pages, LIFO batch:31
[    0.000000] psci: probing for conduit method from ACPI.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] percpu: Embedded 21 pages/cpu @ffff8027ffe6c000 s48000 r8192 d29824 u86016
[    0.000000] pcpu-alloc: s48000 r8192 d29824 u86016 alloc=21*4096
[    0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07
[    0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [0] 14 [0] 15
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] CPU features: enabling workaround for ARM erratum 834220
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 8256527
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: earlycon=hisilpcuart,mmio,0xa01b0000,0,0x2f8 console=ttyS0,115200 initrd=filesystem.cpio.gz acpi=force  pci=pcie_bus_perf
[    0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 61440 bytes
[    0.000000] log_buf_len min size: 16384 bytes
[    0.000000] log_buf_len: 131072 bytes
[    0.000000] early log buf free: 10756(65%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] software IO TLB [mem 0x35c10000-0x39c10000] (64MB) mapped at [ffff800035c10000-ffff800039c0ffff]
[    0.000000] Memory: 32893844K/33550332K available (8508K kernel code, 868K rwdata, 3716K rodata, 1024K init, 287K bss, 640104K reserved, 16384K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffff000000000000 - 0xffff000008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000   (129022 GB)
[    0.000000]       .text : 0xffff000008080000 - 0xffff0000088d0000   (  8512 KB)
[    0.000000]     .rodata : 0xffff0000088d0000 - 0xffff000008c80000   (  3776 KB)
[    0.000000]       .init : 0xffff000008c80000 - 0xffff000008d80000   (  1024 KB)
[    0.000000]       .data : 0xffff000008d80000 - 0xffff000008e59200   (   869 KB)
[    0.000000]        .bss : 0xffff000008e59200 - 0xffff000008ea0fc0   (   288 KB)
[    0.000000]     fixed   : 0xffff7dfffe7fd000 - 0xffff7dfffec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffff7e0000000000 - 0xffff800000000000   (  2048 GB maximum)
[    0.000000]               0xffff7e0000000000 - 0xffff7e00a0000000   (  2560 MB actual)
[    0.000000]     memory  : 0xffff800000000000 - 0xffff802800000000   (163840 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=16, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=16.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=16
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] ITS [mem 0xc6000000-0xc601ffff]
[    0.000000] ITS@0x00000000c6000000: allocated 524288 Devices @27df400000 (flat, esz 8, psz 16K, shr 1)
[    0.000000] ITS@0x00000000c6000000: allocated 2048 Virtual CPUs @27df814000 (flat, esz 8, psz 4K, shr 1)
[    0.000000] ITS@0x00000000c6000000: allocated 512 Interrupt Collections @27df811000 (flat, esz 8, psz 4K, shr 1)
[    0.000000] GIC: using LPI property table @0x00000027df830000
[    0.000000] ITS: Allocated 1792 chunks for LPIs
[    0.000000] GICv3: CPU0: found redistributor 10000 region 0:0x000000004d100000
[    0.000000] CPU0: using LPI pending table @0x00000027df840000
[    0.000000] arch_timer: cp15 timer(s) running at 50.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[    0.000001] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
[    0.009129] Console: colour dummy device 80x25
[    0.014161] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=200000)
[    0.025922] pid_max: default: 32768 minimum: 301
[    0.031178] ACPI: Core revision 20160831
[    0.036722] ACPI: 1 ACPI AML tables successfully acquired and loaded

[    0.045689] Security Framework initialized
[    0.054740] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
[    0.076025] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
[    0.089758] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.097656] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.106805] ASID allocator initialised with 65536 entries
[    0.145006] PCI/MSI: ITS@0xc6000000 domain created
[    0.150462] Platform MSI: ITS@0xc6000000 domain created
[    0.156469] Remapping and enabling EFI services.
[    0.161712]   EFI remap 0x0000000031a00000 => 0000000020000000
[    0.168311]   EFI remap 0x0000000031b60000 => 0000000020070000
[    0.174904]   EFI remap 0x0000000039c10000 => 0000000020080000
[    0.181502]   EFI remap 0x0000000039c50000 => 00000000200c0000
[    0.188100]   EFI remap 0x0000000039cd0000 => 00000000200d0000
[    0.194694]   EFI remap 0x0000000039d50000 => 0000000020150000
[    0.201288]   EFI remap 0x0000000039dc0000 => 00000000201a0000
[    0.207890]   EFI remap 0x0000000039e10000 => 00000000201f0000
[    0.214483]   EFI remap 0x0000000039f20000 => 0000000020300000
[    0.221080]   EFI remap 0x0000000039f90000 => 0000000020370000
[    0.227682]   EFI remap 0x0000000039fe0000 => 00000000203c0000
[    0.234280]   EFI remap 0x000000003a030000 => 0000000020410000
[    0.240873]   EFI remap 0x000000003a080000 => 0000000020460000
[    0.247471]   EFI remap 0x000000003a0d0000 => 00000000204b0000
[    0.254069]   EFI remap 0x000000003f110000 => 0000000020500000
[    0.260663]   EFI remap 0x0000000078000000 => 0000000020530000
[    0.267258]   EFI remap 0x00000000a4000000 => 0000000020600000
[    0.273886]   EFI remap 0x00000000a6000000 => 0000000021600000
[    0.280484]   EFI remap 0x00000000d00e0000 => 0000000021610000
[    0.387578] Detected PIPT I-cache on CPU1
[    0.387593] GICv3: CPU1: found redistributor 10001 region 1:0x000000004d130000
[    0.387624] CPU1: using LPI pending table @0x00000027dc300000
[    0.387639] CPU1: Booted secondary processor [411fd071]
[    0.468292] Detected PIPT I-cache on CPU2
[    0.468300] GICv3: CPU2: found redistributor 10002 region 2:0x000000004d160000
[    0.468330] CPU2: using LPI pending table @0x00000027dc330000
[    0.468340] CPU2: Booted secondary processor [411fd071]
[    0.548998] Detected PIPT I-cache on CPU3
[    0.549006] GICv3: CPU3: found redistributor 10003 region 3:0x000000004d190000
[    0.549035] CPU3: using LPI pending table @0x00000027dc370000
[    0.549046] CPU3: Booted secondary processor [411fd071]
[    0.629654] Detected PIPT I-cache on CPU4
[    0.629667] GICv3: CPU4: found redistributor 10100 region 4:0x000000004d1c0000
[    0.629697] CPU4: using LPI pending table @0x00000027dc3a0000
[    0.629711] CPU4: Booted secondary processor [411fd071]
[    0.710354] Detected PIPT I-cache on CPU5
[    0.710362] GICv3: CPU5: found redistributor 10101 region 5:0x000000004d1f0000
[    0.710391] CPU5: using LPI pending table @0x00000027dc3d0000
[    0.710401] CPU5: Booted secondary processor [411fd071]
[    0.791096] Detected PIPT I-cache on CPU6
[    0.791104] GICv3: CPU6: found redistributor 10102 region 6:0x000000004d220000
[    0.791133] CPU6: using LPI pending table @0x00000027dbc20000
[    0.791143] CPU6: Booted secondary processor [411fd071]
[    0.871802] Detected PIPT I-cache on CPU7
[    0.871810] GICv3: CPU7: found redistributor 10103 region 7:0x000000004d250000
[    0.871839] CPU7: using LPI pending table @0x00000027dbc50000
[    0.871849] CPU7: Booted secondary processor [411fd071]
[    0.952618] Detected PIPT I-cache on CPU8
[    0.952631] GICv3: CPU8: found redistributor 10200 region 8:0x000000004d280000
[    0.952662] CPU8: using LPI pending table @0x00000027dbc80000
[    0.952675] CPU8: Booted secondary processor [411fd071]
[    1.033326] Detected PIPT I-cache on CPU9
[    1.033334] GICv3: CPU9: found redistributor 10201 region 9:0x000000004d2b0000
[    1.033364] CPU9: using LPI pending table @0x00000027dbcc0000
[    1.033374] CPU9: Booted secondary processor [411fd071]
[    1.114050] Detected PIPT I-cache on CPU10
[    1.114059] GICv3: CPU10: found redistributor 10202 region 10:0x000000004d2e0000
[    1.114088] CPU10: using LPI pending table @0x00000027dbcf0000
[    1.114098] CPU10: Booted secondary processor [411fd071]
[    1.194777] Detected PIPT I-cache on CPU11
[    1.194786] GICv3: CPU11: found redistributor 10203 region 11:0x000000004d310000
[    1.194815] CPU11: using LPI pending table @0x00000027dbd30000
[    1.194825] CPU11: Booted secondary processor [411fd071]
[    1.275514] Detected PIPT I-cache on CPU12
[    1.275528] GICv3: CPU12: found redistributor 10300 region 12:0x000000004d340000
[    1.275559] CPU12: using LPI pending table @0x00000027dbd60000
[    1.275572] CPU12: Booted secondary processor [411fd071]
[    1.356225] Detected PIPT I-cache on CPU13
[    1.356233] GICv3: CPU13: found redistributor 10301 region 13:0x000000004d370000
[    1.356263] CPU13: using LPI pending table @0x00000027dbdb0000
[    1.356273] CPU13: Booted secondary processor [411fd071]
[    1.436948] Detected PIPT I-cache on CPU14
[    1.436957] GICv3: CPU14: found redistributor 10302 region 14:0x000000004d3a0000
[    1.436986] CPU14: using LPI pending table @0x00000027dbde0000
[    1.436996] CPU14: Booted secondary processor [411fd071]
[    1.517651] Detected PIPT I-cache on CPU15
[    1.517660] GICv3: CPU15: found redistributor 10303 region 15:0x000000004d3d0000
[    1.517690] CPU15: using LPI pending table @0x00000027dbe10000
[    1.517699] CPU15: Booted secondary processor [411fd071]
[    1.517740] Brought up 16 CPUs
[    1.900501] SMP: Total of 16 processors activated.
[    1.905914] CPU features: detected feature: GIC system register CPU interface
[    1.913979] CPU features: detected feature: 32-bit EL0 Support
[    1.920635] CPU: All CPU(s) started at EL2
[    1.925343] alternatives: patching kernel code
[    1.935331] devtmpfs: initialized
[    1.939440] SMBIOS 3.0.0 present.
[    1.943307] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    1.954592] pinctrl core: initialized pinctrl subsystem
[    1.960951] NET: Registered protocol family 16
[    2.059114] cpuidle: using governor menu
[    2.063620] vdso: 2 pages (1 code @ ffff0000088d7000, 1 data @ ffff000008d84000)
[    2.072051] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    2.080284] DMA: preallocated 256 KiB pool for atomic allocations
[    2.087404] ACPI: bus type PCI registered
[    2.092067] Serial: AMBA PL011 UART driver
[    2.117017] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    2.124855] ACPI: Added _OSI(Module Device)
[    2.129611] ACPI: Added _OSI(Processor Device)
[    2.134659] ACPI: Added _OSI(3.0 _SCP Extensions)
[    2.140012] ACPI: Added _OSI(Processor Aggregator Device)
[    2.148077] ACPI: Interpreter enabled
[    2.152240] ACPI: Using GIC for interrupt routing
[    2.157602] ACPI: MCFG table detected, 3 entries
[    2.180043] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
[    2.187080] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    2.196543] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME AER]
[    2.205282] acpi PNP0A08:00: _OSC: OS now controls [PCIeCapability]
[    2.212420] acpi PNP0A08:00: MCFG quirk: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f] with hisi_pcie_ops
[    2.224524] acpi PNP0A08:00: ECAM area [mem 0xb0000000-0xb1ffffff] reserved by HISI0081:00
[    2.234149] acpi PNP0A08:00: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f]
[    2.242587] Remapped I/O 0x00000000b7ff0000 to [io  0x0000-0xffff window]
[    2.250473] PCI host bridge to bus 0000:00
[    2.255207] pci_bus 0000:00: root bus resource [mem 0xb2000000-0xb7feffff window]
[    2.263838] pci_bus 0000:00: root bus resource [io  0x0000-0xffff window]
[    2.271688] pci_bus 0000:00: root bus resource [bus 00-1f]
[    2.278039] pci 0000:00:00.0: [19e5:1610] type 01 class 0x060400
[    2.278049] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x0000ffff]
[    2.278087] pci 0000:00:00.0: supports D1 D2
[    2.278089] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.329441] pci 0000:00:00.0: BAR 0: assigned [mem 0xb2000000-0xb200ffff]
[    2.337272] pci 0000:00:00.0: PCI bridge to [bus 01]
[    2.343009] pci 0000:00:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.352767] ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus e0-ff])
[    2.359894] acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    2.369463] acpi PNP0A08:01: _OSC: platform does not support [PCIeHotplug PME AER]
[    2.378315] acpi PNP0A08:01: _OSC: OS now controls [PCIeCapability]
[    2.385532] acpi PNP0A08:01: MCFG quirk: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff] with hisi_pcie_ops
[    2.397799] acpi PNP0A08:01: ECAM area [mem 0xbe000000-0xbfffffff] reserved by HISI0081:01
[    2.407538] acpi PNP0A08:01: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff]
[    2.416108] Remapped I/O 0x00000000bdff0000 to [io  0x10000-0x1ffff window]
[    2.424194] PCI host bridge to bus 0001:e0
[    2.428905] pci_bus 0001:e0: root bus resource [mem 0xb8000000-0xbdfeffff window]
[    2.437533] pci_bus 0001:e0: root bus resource [io  0x10000-0x1ffff window] (bus address [0x0000-0xffff])
[    2.448566] pci_bus 0001:e0: root bus resource [bus e0-ff]
[    2.454894] pci 0001:e0:00.0: [19e5:1610] type 01 class 0x060400
[    2.454902] pci 0001:e0:00.0: reg 0x10: [mem 0x00000000-0x00000fff]
[    2.454910] pci 0001:e0:00.0: reg 0x38: [mem 0xfffe0000-0xffffffff pref]
[    2.454938] pci 0001:e0:00.0: supports D1 D2
[    2.454940] pci 0001:e0:00.0: PME# supported from D0 D1 D3hot
[    2.455070] pci 0001:e1:00.0: [1000:005d] type 00 class 0x010400
[    2.455079] pci 0001:e1:00.0: reg 0x10: [io  0x10000-0x100ff]
[    2.455086] pci 0001:e1:00.0: reg 0x14: [mem 0xb8100000-0xb810ffff 64bit]
[    2.455093] pci 0001:e1:00.0: reg 0x1c: [mem 0xb8000000-0xb80fffff 64bit]
[    2.455101] pci 0001:e1:00.0: reg 0x30: [mem 0xfff00000-0xffffffff pref]
[    2.455151] pci 0001:e1:00.0: supports D1 D2
[    2.466911] pci 0001:e0:00.0: BAR 14: assigned [mem 0xb8000000-0xb82fffff]
[    2.474842] pci 0001:e0:00.0: BAR 6: assigned [mem 0xb8300000-0xb831ffff pref]
[    2.483164] pci 0001:e0:00.0: BAR 0: assigned [mem 0xb8320000-0xb8320fff]
[    2.490996] pci 0001:e0:00.0: BAR 13: assigned [io  0x10000-0x10fff]
[    2.498325] pci 0001:e1:00.0: BAR 3: assigned [mem 0xb8000000-0xb80fffff 64bit]
[    2.506753] pci 0001:e1:00.0: BAR 6: assigned [mem 0xb8100000-0xb81fffff pref]
[    2.515089] pci 0001:e1:00.0: BAR 1: assigned [mem 0xb8200000-0xb820ffff 64bit]
[    2.523513] pci 0001:e1:00.0: BAR 0: assigned [io  0x10000-0x100ff]
[    2.530749] pci 0001:e0:00.0: PCI bridge to [bus e1]
[    2.536476] pci 0001:e0:00.0:   bridge window [io  0x10000-0x10fff]
[    2.543720] pci 0001:e0:00.0:   bridge window [mem 0xb8000000-0xb82fffff]
[    2.551552] pci 0001:e0:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.561283] pci 0001:e1:00.0: Max Payload Size set to  512/4096 (was  128), Max Read Rq  512
[    2.571060] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 80-9f])
[    2.578184] acpi PNP0A08:02: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    2.587752] acpi PNP0A08:02: _OSC: platform does not support [PCIeHotplug PME AER]
[    2.596595] acpi PNP0A08:02: _OSC: OS now controls [PCIeCapability]
[    2.603816] acpi PNP0A08:02: MCFG quirk: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f] with hisi_pcie_ops
[    2.616078] acpi PNP0A08:02: ECAM area [mem 0xa8000000-0xa9ffffff] reserved by HISI0081:02
[    2.625841] acpi PNP0A08:02: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f]
[    2.634385] Remapped I/O 0x00000000afff0000 to [io  0x20000-0x2ffff window]
[    2.642469] PCI host bridge to bus 0002:80
[    2.647189] pci_bus 0002:80: root bus resource [mem 0xaa000000-0xaffeffff window]
[    2.655820] pci_bus 0002:80: root bus resource [io  0x20000-0x2ffff window] (bus address [0x0000-0xffff])
[    2.666862] pci_bus 0002:80: root bus resource [bus 80-9f]
[    2.673199] pci 0002:80:00.0: [19e5:1610] type 01 class 0x060400
[    2.673207] pci 0002:80:00.0: reg 0x10: [mem 0x00000000-0x0000ffff]
[    2.673237] pci 0002:80:00.0: supports D1 D2
[    2.673239] pci 0002:80:00.0: PME# supported from D0 D1 D3hot
[    2.673370] pci 0002:81:00.0: [10b5:8725] type 01 class 0x060400
[    2.673381] pci 0002:81:00.0: reg 0x10: [mem 0xaa200000-0xaa23ffff]
[    2.673453] pci 0002:81:00.0: PME# supported from D0 D3hot D3cold
[    2.685262] pci 0002:82:00.0: [10b5:8725] type 01 class 0x060400
[    2.685343] pci 0002:82:00.0: PME# supported from D0 D3hot D3cold
[    2.685444] pci 0002:82:01.0: [10b5:8725] type 01 class 0x060400
[    2.685525] pci 0002:82:01.0: PME# supported from D0 D3hot D3cold
[    2.685619] pci 0002:82:02.0: [10b5:8725] type 01 class 0x060400
[    2.685699] pci 0002:82:02.0: PME# supported from D0 D3hot D3cold
[    2.685800] pci 0002:82:08.0: [10b5:8725] type 01 class 0x060400
[    2.685880] pci 0002:82:08.0: PME# supported from D0 D3hot D3cold
[    2.686094] pci 0002:84:00.0: [19e5:1711] type 00 class 0x030000
[    2.686112] pci 0002:84:00.0: reg 0x10: [mem 0xac000000-0xadffffff pref]
[    2.686122] pci 0002:84:00.0: reg 0x14: [mem 0xaa000000-0xaa1fffff]
[    2.686221] pci 0002:84:00.0: supports D1
[    2.686223] pci 0002:84:00.0: PME# supported from D0 D1 D3hot
[    2.697355] pci 0002:80:00.0: BAR 14: assigned [mem 0xaa000000-0xae7fffff]
[    2.705290] pci 0002:80:00.0: BAR 15: assigned [mem 0xae800000-0xaebfffff 64bit pref]
[    2.714321] pci 0002:80:00.0: BAR 0: assigned [mem 0xaec00000-0xaec0ffff]
[    2.722153] pci 0002:80:00.0: BAR 13: assigned [io  0x20000-0x21fff]
[    2.729469] pci 0002:81:00.0: BAR 14: assigned [mem 0xaa000000-0xadffffff]
[    2.737392] pci 0002:81:00.0: BAR 15: assigned [mem 0xae800000-0xaebfffff 64bit pref]
[    2.746429] pci 0002:81:00.0: BAR 0: assigned [mem 0xae000000-0xae03ffff]
[    2.754264] pci 0002:81:00.0: BAR 13: assigned [io  0x20000-0x21fff]
[    2.761587] pci 0002:82:01.0: BAR 14: assigned [mem 0xaa000000-0xacffffff]
[    2.769512] pci 0002:82:00.0: BAR 14: assigned [mem 0xad000000-0xad1fffff]
[    2.777419] pci 0002:82:00.0: BAR 15: assigned [mem 0xae800000-0xae9fffff 64bit pref]
[    2.786431] pci 0002:82:01.0: BAR 15: assigned [mem 0xaea00000-0xaebfffff 64bit pref]
[    2.795460] pci 0002:82:00.0: BAR 13: assigned [io  0x20000-0x20fff]
[    2.802783] pci 0002:82:01.0: BAR 13: assigned [io  0x21000-0x21fff]
[    2.810091] pci 0002:82:00.0: PCI bridge to [bus 83]
[    2.815813] pci 0002:82:00.0:   bridge window [io  0x20000-0x20fff]
[    2.823022] pci 0002:82:00.0:   bridge window [mem 0xad000000-0xad1fffff]
[    2.830844] pci 0002:82:00.0:   bridge window [mem 0xae800000-0xae9fffff 64bit pref]
[    2.839754] pci 0002:84:00.0: BAR 0: assigned [mem 0xaa000000-0xabffffff pref]
[    2.848074] pci 0002:84:00.0: BAR 1: assigned [mem 0xac000000-0xac1fffff]
[    2.855892] pci 0002:82:01.0: PCI bridge to [bus 84]
[    2.861609] pci 0002:82:01.0:   bridge window [io  0x21000-0x21fff]
[    2.868824] pci 0002:82:01.0:   bridge window [mem 0xaa000000-0xacffffff]
[    2.876641] pci 0002:82:01.0:   bridge window [mem 0xaea00000-0xaebfffff 64bit pref]
[    2.885552] pci 0002:82:02.0: PCI bridge to [bus 85]
[    2.891272] pci 0002:82:08.0: PCI bridge to [bus 86]
[    2.896987] pci 0002:81:00.0: PCI bridge to [bus 82-86]
[    2.903005] pci 0002:81:00.0:   bridge window [io  0x20000-0x21fff]
[    2.910234] pci 0002:81:00.0:   bridge window [mem 0xaa000000-0xadffffff]
[    2.918067] pci 0002:81:00.0:   bridge window [mem 0xae800000-0xaebfffff 64bit pref]
[    2.926998] pci 0002:80:00.0: PCI bridge to [bus 81-86]
[    2.933025] pci 0002:80:00.0:   bridge window [io  0x20000-0x21fff]
[    2.940243] pci 0002:80:00.0:   bridge window [mem 0xaa000000-0xae7fffff]
[    2.948075] pci 0002:80:00.0:   bridge window [mem 0xae800000-0xaebfffff 64bit pref]
[    2.957001] pci 0002:80:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.966723] pci 0002:81:00.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.976447] pci 0002:82:00.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.986191] pci 0002:82:01.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.995931] pci 0002:84:00.0: Max Payload Size set to  256/ 256 (was  128), Max Read Rq  256
[    3.005655] pci 0002:82:02.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    3.015393] pci 0002:82:08.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    3.025549] vgaarb: device added: PCI:0002:84:00.0,decodes=io+mem,owns=none,locks=none
[    3.034497] vgaarb: loaded
[    3.037549] vgaarb: bridge control possible 0002:84:00.0
[    3.043694] SCSI subsystem initialized
[    3.048016] libata version 3.00 loaded.
[    3.048093] ACPI: bus type USB registered
[    3.052669] usbcore: registered new interface driver usbfs
[    3.058888] usbcore: registered new interface driver hub
[    3.064941] usbcore: registered new device driver usb
[    3.070764] pps_core: LinuxPPS API ver. 1 registered
[    3.076374] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    3.086699] PTP clock support registered
[    3.091204] Registered efivars operations
[    3.095935] Advanced Linux Sound Architecture Driver Initialized.
[    3.103198] clocksource: Switched to clocksource arch_sys_counter
[    3.110185] VFS: Disk quotas dquot_6.6.0
[    3.114673] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    3.122588] pnp: PnP ACPI init
[    3.126233] system 00:00: [mem 0xa0090000-0xa009ffff] has been reserved
[    3.133740] system 00:00: [mem 0xb0000000-0xb1ffffff] could not be reserved
[    3.141639] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.141704] system 00:01: [mem 0xa0200000-0xa020ffff] has been reserved
[    3.149222] system 00:01: [mem 0xbe000000-0xbfffffff] could not be reserved
[    3.157139] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.157202] system 00:02: [mem 0xa00a0000-0xa00affff] has been reserved
[    3.164710] system 00:02: [mem 0xa8000000-0xa9ffffff] could not be reserved
[    3.172604] system 00:02: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.172612] pnp: PnP ACPI: found 3 devices
[    3.180083] NET: Registered protocol family 2
[    3.185559] TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
[    3.194711] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    3.202681] TCP: Hash tables configured (established 262144 bind 65536)
[    3.210322] UDP hash table entries: 16384 (order: 7, 524288 bytes)
[    3.217564] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
[    3.225356] NET: Registered protocol family 1
[    3.230488] RPC: Registered named UNIX socket transport module.
[    3.237217] RPC: Registered udp transport module.
[    3.242565] RPC: Registered tcp transport module.
[    3.247908] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    3.255230] pci 0001:e1:00.0: [Firmware Bug]: VPD access disabled
[    3.262162] PCI: CLS 0 bytes, default 128
[    3.262252] Unpacking initramfs...
[    3.649014] Freeing initrd memory: 27492K (ffff80001e520000 - ffff80001fff9000)
[    3.657578] kvm [1]: 8-bit VMID
[    3.661145] kvm [1]: IDMAP page: 8c6000
[    3.665489] kvm [1]: HYP VA range: 800000000000:ffffffffffff
[    3.672639] kvm [1]: Hyp mode initialized successfully
[    3.678492] kvm [1]: vgic-v2@fe020000
[    3.682672] kvm [1]: GIC system register CPU interface enabled
[    3.689472] kvm [1]: vgic interrupt IRQ1
[    3.693959] kvm [1]: virtual timer IRQ4
[    3.699944] futex hash table entries: 4096 (order: 7, 524288 bytes)
[    3.707148] audit: initializing netlink subsys (disabled)
[    3.713283] audit: type=2000 audit(3.068:1): initialized
[    3.719528] workingset: timestamp_bits=44 max_order=23 bucket_order=0
[    3.730723] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    3.737669] NFS: Registering the id_resolver key type
[    3.743390] Key type id_resolver registered
[    3.748115] Key type id_legacy registered
[    3.752648] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    3.760285] 9p: Installing v9fs 9p2000 file system support
[    3.767646] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[    3.776005] io scheduler noop registered
[    3.780493] io scheduler cfq registered (default)
[    3.786034] libphy: mdio_driver_register: phy-bcm-ns2-pci
[    3.792498] pcieport 0000:00:00.0: can't derive routing for PCI INT A
[    3.799778] pcieport 0000:00:00.0: PCI INT A: no GSI
[    3.805440] pcieport 0001:e0:00.0: can't derive routing for PCI INT A
[    3.812736] pcieport 0001:e0:00.0: PCI INT A: no GSI
[    3.818379] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.825658] pcieport 0002:80:00.0: PCI INT A: no GSI
[    3.831294] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.838569] pcieport 0002:81:00.0: PCI INT A: no GSI
[    3.844358] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.851639] pcieport 0002:82:00.0: PCI INT A: no GSI
[    3.857395] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.864672] pcieport 0002:82:01.0: PCI INT A: no GSI
[    3.870426] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.877706] pcieport 0002:82:02.0: PCI INT A: no GSI
[    3.883456] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.890735] pcieport 0002:82:08.0: PCI INT A: no GSI
[    3.896709] ipmi message handler version 39.2
[    3.901658] ipmi device interface
[    3.905458] IPMI System Interface driver.
[    3.910064] ipmi_si: probing via SMBIOS
[    3.914404] ipmi_si: SMBIOS: io 0xe4 regsize 1 spacing 1 irq 0
[    3.921008] ipmi_si: Adding SMBIOS-specified bt state machine
[    3.927549] ipmi_si: Trying SMBIOS-specified bt state machine at i/o address 0xe4, slave address 0x0, irq 0
[    3.995245] IPMI BT: using default values
[    3.999783] IPMI BT: req2rsp=5 secs retries=2
[    4.207340] ipmi_si ipmi_si.0: Found new BMC (man_id: 0x0007db, prod_id: 0x0001, dev_id: 0x01)
[    4.217181] ipmi_si ipmi_si.0: IPMI bt interface initialized
[    4.224462] xenfs: not registering filesystem on non-xen platform
[    4.232736] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    4.240519] console [ttyS0] disabled
[    4.265944] serial8250.10: ttyS0 at I/O 0x2f8 (irq = 0, base_baud = 115200) is a 16550A
[    4.275069] console [ttyS0] enabled
[    4.282512] bootconsole [hisilpcuart0] disabled
[    4.292350] SuperH (H)SCI(F) driver initialized
[    4.296951] msm_serial: driver initialized
[    4.301280] Failed to find cpu0 device node
[    4.305463] Unable to detect cache hierarchy from DT for CPU 0
[    4.314331] loop: module loaded
[    4.317688] hisi_sas: driver version v1.6
[    4.455735] scsi host0: hisi_sas
[    4.591730] scsi host1: hisi_sas
[    4.727757] scsi host2: hisi_sas
[    4.732451] libphy: Fixed MDIO Bus: probed
[    4.736690] tun: Universal TUN/TAP device driver, 1.6
[    4.741742] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    4.827207] libphy: Hisilicon MII Bus: probed
[    5.707286] netif_napi_add() called with weight 256 on device eth%d
[    5.803822] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    5.809652] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    5.815603] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    5.822548] igb: Copyright (c) 2007-2014 Intel Corporation.
[    5.828140] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
[    5.835950] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    5.841882] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 4.4.0-k
[    5.849518] ixgbe: Copyright (c) 1999-2016 Intel Corporation.
[    5.855328] sky2: driver version 1.30
[    5.859170] VFIO - User Level meta-driver version: 0.3
[    5.865063] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    5.871579] ehci-pci: EHCI PCI platform driver
[    5.876044] ehci-platform: EHCI generic platform driver
[    5.881310] ehci-platform PNP0D20:00: EHCI Host Controller
[    5.886798] ehci-platform PNP0D20:00: new USB bus registered, assigned bus number 1
[    5.894545] ehci-platform PNP0D20:00: irq 6, io mem 0xa7020000
[    5.915203] ehci-platform PNP0D20:00: USB 2.0 started, EHCI 1.00
[    5.921487] hub 1-0:1.0: USB hub found
[    5.925247] hub 1-0:1.0: 2 ports detected
[    5.929487] ehci-exynos: EHCI EXYNOS driver
[    5.933695] ehci-msm: Qualcomm On-Chip EHCI Host Controller
[    5.939287] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    5.945462] ohci-pci: OHCI PCI platform driver
[    5.949932] ohci-platform: OHCI generic platform driver
[    5.955171] ohci-exynos: OHCI EXYNOS driver
[    5.959468] usbcore: registered new interface driver usb-storage
[    5.965733] mousedev: PS/2 mouse device common for all mice
[    6.097475] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
[    6.103714] i2c /dev entries driver
[    6.107668] sdhci: Secure Digital Host Controller Interface driver
[    6.113839] sdhci: Copyright(c) Pierre Ossman
[    6.118253] Synopsys Designware Multimedia Card Interface Driver
[    6.124326] sdhci-pltfm: SDHCI platform and OF driver helper
[    6.130265] ledtrig-cpu: registered to indicate activity on CPUs
[    6.136571] usbcore: registered new interface driver usbhid
[    6.142149] usbhid: USB HID core driver
[    6.146490] NET: Registered protocol family 17
[    6.150980] 9pnet: Installing 9P2000 support
[    6.155281] Key type dns_resolver registered
[    6.159799] registered taskstats version 1
[    6.227038] rtc-efi rtc-efi: setting system clock to 2016-12-06 11:30:25 UTC (1481023825)
[    6.235251] ALSA device list:
[    6.238219]   No soundcards found.
[    6.242270] Freeing unused kernel memory: 1024K (ffff800000c80000 - ffff800000d80000)
[    6.323209] usb 1-1: new high-speed USB device number 2 using ehci-platform
[    6.475822] hub 1-1:1.0: USB hub found
[    6.479639] hub 1-1:1.0: 4 ports detected
[    6.603202] usb 1-2: new high-speed USB device number 3 using ehci-platform
[    6.755814] hub 1-2:1.0: USB hub found
[    6.759639] hub 1-2:1.0: 4 ports detected
[    7.051201] usb 1-2.1: new full-speed USB device number 4 using ehci-platform
[    7.167848] input: Keyboard/Mouse KVM 1.1.0 as /devices/platform/PNP0D20:00/usb1/1-2/1-2.1/1-2.1:1.0/0003:12D1:0003.0001/input/input0
[    7.239434] hid-generic 0003:12D1:0003.0001: input: USB HID v1.10 Keyboard [Keyboard/Mouse KVM 1.1.0] on usb-PNP0D20:00-2.1/input0
[    7.252437] input: Keyboard/Mouse KVM 1.1.0 as /devices/platform/PNP0D20:00/usb1/1-2/1-2.1/1-2.1:1.1/0003:12D1:0003.0002/input/input1
[    7.264494] hid-generic 0003:12D1:0003.0002: input: USB HID v1.10 Mouse [Keyboard/Mouse KVM 1.1.0] on usb-PNP0D20:00-2.1/input1
root@(none)$ cat /proc/iomem
00000000-00024fff : reserved
00026000-319fffff : System RAM
   00080000-00c7ffff : Kernel code
   00d80000-00ea6fff : Kernel data
31a00000-31a6ffff : reserved
31a70000-31b5ffff : System RAM
31b60000-31b6ffff : reserved
31b70000-31b7dfff : System RAM
31b7e000-31b81fff : reserved
31b82000-39c0ffff : System RAM
39c10000-3a11ffff : reserved
3a120000-3f10ffff : System RAM
3f110000-3f13ffff : reserved
3f140000-3fbfffff : System RAM
603c0000-603cffff : HISI0141:00
   603c0000-603cffff : HISI0141:00
a0080000-a008ffff : HISI0152:02
   a0080000-a008ffff : HISI0152:01
     a0080000-a008ffff : HISI0152:00
a0090000-a009ffff : pnp 00:00
a00a0000-a00affff : pnp 00:02
a01b0000-a01b0fff : HISI0191:00
   a01b0000-a01b0fff : HISI0191:00
a0200000-a020ffff : pnp 00:01
a2000000-a200ffff : HISI0162:01
   a2000000-a200ffff : HISI0162:01
a3000000-a300ffff : HISI0162:02
   a3000000-a300ffff : HISI0162:02
a7020000-a702ffff : PNP0D20:00
   a7020000-a702ffff : PNP0D20:00
a8000000-a9ffffff : PCI ECAM
aa000000-affeffff : PCI Bus 0002:80
   aa000000-ae7fffff : PCI Bus 0002:81
     aa000000-adffffff : PCI Bus 0002:82
       aa000000-acffffff : PCI Bus 0002:84
         aa000000-abffffff : 0002:84:00.0
         ac000000-ac1fffff : 0002:84:00.0
       ad000000-ad1fffff : PCI Bus 0002:83
     ae000000-ae03ffff : 0002:81:00.0
   ae800000-aebfffff : PCI Bus 0002:81
     ae800000-aebfffff : PCI Bus 0002:82
       ae800000-ae9fffff : PCI Bus 0002:83
       aea00000-aebfffff : PCI Bus 0002:84
   aec00000-aec0ffff : 0002:80:00.0
b0000000-b1ffffff : PCI ECAM
b2000000-b7feffff : PCI Bus 0000:00
   b2000000-b200ffff : 0000:00:00.0
b8000000-bdfeffff : PCI Bus 0001:e0
   b8000000-b82fffff : PCI Bus 0001:e1
     b8000000-b80fffff : 0001:e1:00.0
     b8100000-b81fffff : 0001:e1:00.0
     b8200000-b820ffff : 0001:e1:00.0
   b8300000-b831ffff : 0001:e0:00.0
   b8320000-b8320fff : 0001:e0:00.0
be000000-bfffffff : PCI ECAM
c0080000-c008ffff : HISI0152:05
   c0080000-c008ffff : HISI0152:04
     c0080000-c008ffff : HISI0152:03
c3000000-c300ffff : HISI0162:00
   c3000000-c300ffff : HISI0162:00
c4000000-c40fffff : HISI00D1:00
c5000000-c588ffff : HISI00B2:00
   c5000000-c588ffff : HISI00B2:00
c7000000-c705ffff : HISI00B2:00
   c7000000-c705ffff : HISI00B2:00
2040000000-27ffffffff : System RAM

>
> Changes from v10 to v11:
>
> - Discard "consumer" resources from PNP0A03 host bridge windows (arm64
>   only).  This allows using consumer PNP0A03 descriptors (rather than the
>   PNP0C02 workaround used on x86) for bridge register space.
>
> - Add APM X-Gene MCFG quirks.
>
> - If ACPI and quirks are enabled, always build ThunderX, X-Gene, and
>   HiSilicon drivers on ARM64.  Add internal ifdefs so we only compile the
>   parts necessary for ACPI (the ECAM init and accessors).
>
> - s/node/seg/ in ThunderX MCFG quirk macros.
>
> - Move ECAM check for ACPI device reservation from pci_ecam_create() to
>   that arm64-specific pci_acpi_setup_ecam_mapping().
>
> - Add local "struct device *dev" pointers for brevity.
>
> - Miscellaneous build fixes.
>
> ---
>
> Bjorn Helgaas (6):
>       ACPI: Add acpi_resource_consumer() to find device that claims a resource
>       x86/PCI: Use acpi_resource_consumer() to search ACPI namespace for MMCFG
>       arm64: PCI: Add local struct device pointers
>       arm64: PCI: Search ACPI namespace to ensure ECAM space is reserved
>       arm64: PCI: Exclude ACPI "consumer" resources from host bridge windows
>       PCI: thunder-pem: Factor out resource lookup
>
> Christopher Covington (1):
>       PCI: Add MCFG quirks for Qualcomm QDF2432 host controller
>
> Dongdong Liu (2):
>       PCI/ACPI: Provide acpi_get_rc_resources() for ARM64 platform
>       PCI: Add MCFG quirks for HiSilicon Hip05/06/07 host controllers
>
> Duc Dang (1):
>       PCI: Add MCFG quirks for X-Gene host controller
>
> Tomasz Nowicki (5):
>       arm64: PCI: Manage controller-specific data on per-controller basis
>       PCI/ACPI: Extend pci_mcfg_lookup() to return ECAM config accessors
>       PCI/ACPI: Check for platform-specific MCFG quirks
>       PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller
>       PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller
>
>
>  arch/arm64/kernel/pci.c             |   68 ++++++++-----
>  arch/x86/pci/mmconfig-shared.c      |   69 ++-----------
>  drivers/acpi/pci_mcfg.c             |  190 ++++++++++++++++++++++++++++++++++-
>  drivers/acpi/resource.c             |   57 +++++++++++
>  drivers/pci/ecam.c                  |   12 ++
>  drivers/pci/host/Kconfig            |   10 +-
>  drivers/pci/host/Makefile           |    8 +
>  drivers/pci/host/pci-thunder-ecam.c |    9 +-
>  drivers/pci/host/pci-thunder-pem.c  |   94 +++++++++++++----
>  drivers/pci/host/pci-xgene.c        |  126 ++++++++++++++++++++++-
>  drivers/pci/host/pcie-hisi.c        |  101 +++++++++++++++++++
>  drivers/pci/pci-acpi.c              |   76 ++++++++++++++
>  drivers/pci/pci.h                   |    5 +
>  include/linux/acpi.h                |    7 +
>  include/linux/pci-acpi.h            |    4 +
>  include/linux/pci-ecam.h            |    9 ++
>  16 files changed, 718 insertions(+), 127 deletions(-)
>
> .
>

Re: [PATCH v11 00/15] PCI: ARM64 ECAM quirks

[Tomasz Nowicki]

On 06.12.2016 00:25, Bjorn Helgaas wrote:
> Here's another try.  The biggest change is to add the APM X-Gene quirks.
>
> These are on my pci/ecam branch.  Please test and report any issues.
> Please also collect dmesg and /proc/iomem contents and I'll try to
> summarize the current ACPI firmware situation on ARM64 and update the
> writeup with recommendations for future platforms.
>
> Changes from v10 to v11:
>
> - Discard "consumer" resources from PNP0A03 host bridge windows (arm64
>   only).  This allows using consumer PNP0A03 descriptors (rather than the
>   PNP0C02 workaround used on x86) for bridge register space.
>
> - Add APM X-Gene MCFG quirks.
>
> - If ACPI and quirks are enabled, always build ThunderX, X-Gene, and
>   HiSilicon drivers on ARM64.  Add internal ifdefs so we only compile the
>   parts necessary for ACPI (the ECAM init and accessors).
>
> - s/node/seg/ in ThunderX MCFG quirk macros.
>
> - Move ECAM check for ACPI device reservation from pci_ecam_create() to
>   that arm64-specific pci_acpi_setup_ecam_mapping().
>
> - Add local "struct device *dev" pointers for brevity.
>
> - Miscellaneous build fixes.
>

The dmesg and /proc/iomem for ThunderX pass2.0 1-socket machine:

EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
Booting Linux on physical CPU 0x0
Linux version 4.9.0-rc8+ (tn@tn-desktop) (gcc version 5.2.0 (Cavium Inc. 
Version 0.99 build 401) ) #33 SMP PREEMPT Tue Dec 6 13:57:46 CET 2016
Boot CPU: AArch64 Processor [431f0a10]
earlycon: pl11 at MMIO 0x000087e024000000 (options '')
bootconsole [pl11] enabled
debug: ignoring loglevel setting.
efi: Getting EFI parameters from FDT:
efi: EFI v2.40 by Cavium Thunder cn88xx EFI 
ThunderX-Firmware-Release-1.22.12-build_04-2-g3ca4638-dirty Dec  6 2016 1
efi:  ACPI=0xfffff000  ACPI 2.0=0xfffff014  SMBIOS 3.0=0x1ffaa58000
ACPI: Early table checksum verification disabled
ACPI: RSDP 0x00000000FFFFF014 000024 (v02 CAVIUM)
ACPI: XSDT 0x00000000FFFFE0E8 00006C (v01 CAVIUM THUNDERX 00000000 01000013)
ACPI: FACP 0x00000000FFFF7000 000114 (v06 CAVIUM THUNDERX 00000000 INTL 
20150717)
ACPI: DSDT 0x00000000FFFF8000 0020C5 (v02 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: APIC 0x00000000FFFFD000 000F68 (v03 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: SSDT 0x00000000FFFFC000 000158 (v02 CAVIUM CRB-1S   00000001 INTL 
20150717)
ACPI: DBG2 0x00000000FFFFB000 0000AA (v01 CAVIUM CN88XDBG 00000000 INTL 
20150717)
ACPI: GTDT 0x00000000FFFF5000 0000E0 (v02 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: IORT 0x00000000FFFF3000 0013D4 (v01 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: MCFG 0x00000000FFFF2000 00006C (v01 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: SLIT 0x00000000FFFF1000 000030 (v01 CAVIUM TEMPLATE 00000001 INTL 
20150717)
ACPI: SPCR 0x00000000FFFF0000 000050 (v02 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: SPCR: console: pl011,mmio,0x87e024000000,115200
ACPI: NUMA: Failed to initialise from firmware
NUMA: Faking a node at [mem 0x0000000000000000-0x0000001fffffffff]
NUMA: Adding memblock [0x1400000 - 0xfffeffff] on node 0
NUMA: Adding memblock [0xffff0000 - 0xffffffff] on node 0
NUMA: Adding memblock [0x100000000 - 0x1ffaa4ffff] on node 0
NUMA: Adding memblock [0x1ffaa50000 - 0x1ffaa7ffff] on node 0
NUMA: Adding memblock [0x1ffaa80000 - 0x1fffa3ffff] on node 0
NUMA: Adding memblock [0x1fffa40000 - 0x1fffa8ffff] on node 0
NUMA: Adding memblock [0x1fffa90000 - 0x1fffffffff] on node 0
NUMA: Initmem setup node 0 [mem 0x01400000-0x1fffffffff]
NUMA: NODE_DATA [mem 0x1fffffe200-0x1fffffffff]
Zone ranges:
   DMA      [mem 0x0000000001400000-0x00000000ffffffff]
   Normal   [mem 0x0000000100000000-0x0000001fffffffff]
Movable zone start for each node
Early memory node ranges
   node   0: [mem 0x0000000001400000-0x00000000fffeffff]
   node   0: [mem 0x00000000ffff0000-0x00000000ffffffff]
   node   0: [mem 0x0000000100000000-0x0000001ffaa4ffff]
   node   0: [mem 0x0000001ffaa50000-0x0000001ffaa7ffff]
   node   0: [mem 0x0000001ffaa80000-0x0000001fffa3ffff]
   node   0: [mem 0x0000001fffa40000-0x0000001fffa8ffff]
   node   0: [mem 0x0000001fffa90000-0x0000001fffffffff]
Initmem setup node 0 [mem 0x0000000001400000-0x0000001fffffffff]
On node 0 totalpages: 2096832
   DMA zone: 64 pages used for memmap
   DMA zone: 0 pages reserved
   DMA zone: 65216 pages, LIFO batch:1
   Normal zone: 1984 pages used for memmap
   Normal zone: 2031616 pages, LIFO batch:1
psci: probing for conduit method from ACPI.
psci: PSCIv0.2 detected in firmware.
psci: Using standard PSCI v0.2 function IDs
psci: Trusted OS resident on physical CPU 0x0
percpu: Embedded 2 pages/cpu @ffff801ff76f0000 s48256 r8192 d74624 u131072
pcpu-alloc: s48256 r8192 d74624 u131072 alloc=2*65536
pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07
pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [0] 14 [0] 15
pcpu-alloc: [0] 16 [0] 17 [0] 18 [0] 19 [0] 20 [0] 21 [0] 22 [0] 23
pcpu-alloc: [0] 24 [0] 25 [0] 26 [0] 27 [0] 28 [0] 29 [0] 30 [0] 31
pcpu-alloc: [0] 32 [0] 33 [0] 34 [0] 35 [0] 36 [0] 37 [0] 38 [0] 39
pcpu-alloc: [0] 40 [0] 41 [0] 42 [0] 43 [0] 44 [0] 45 [0] 46 [0] 47
Detected VIPT I-cache on CPU0
CPU features: enabling workaround for Cavium erratum 27456
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 2094784
Policy zone: Normal
Kernel command line: BOOT_IMAGE=(tftp)efi/Image_tn root=/dev/sda2 rw 
console=ttyAMA0 earlycon=pl011,0x87e024000000 kgdboc=ttyAMA0,115200 
uefi_debug acpi=force ignore_loglevel
log_buf_len individual max cpu contribution: 4096 bytes
log_buf_len total cpu_extra contributions: 192512 bytes
log_buf_len min size: 16384 bytes
log_buf_len: 262144 bytes
early log buf free: 10832(66%)
PID hash table entries: 4096 (order: -1, 32768 bytes)
software IO TLB [mem 0xfbfe0000-0xfffe0000] (64MB) mapped at 
[ffff8000fbfe0000-ffff8000fffdffff]
Memory: 133978304K/134197248K available (8572K kernel code, 724K rwdata, 
3136K rodata, 640K init, 645K bss, 218944K reserved, 0K cma-reserved)

Virtual kernel memory layout:
     modules : 0xffff000000000000 - 0xffff000008000000   (   128 MB)
     vmalloc : 0xffff000008000000 - 0xffff7bdfffff0000   (126847 GB)
       .text : 0xffff000008080000 - 0xffff0000088e0000   (  8576 KB)
     .rodata : 0xffff0000088e0000 - 0xffff000008c00000   (  3200 KB)
       .init : 0xffff000008c00000 - 0xffff000008ca0000   (   640 KB)
       .data : 0xffff000008ca0000 - 0xffff000008d55200   (   725 KB)
        .bss : 0xffff000008d55200 - 0xffff000008df6968   (   646 KB)
     fixed   : 0xffff7fdffe7d0000 - 0xffff7fdffec00000   (  4288 KB)
     PCI I/O : 0xffff7fdffee00000 - 0xffff7fdfffe00000   (    16 MB)
     vmemmap : 0xffff7fe000000000 - 0xffff800000000000   (   128 GB maximum)
               0xffff7fe000005000 - 0xffff7fe008000000   (   127 MB actual)
     memory  : 0xffff800001400000 - 0xffff802000000000   (131052 MB)
SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=48, Nodes=1
Preemptible hierarchical RCU implementation.

RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=48
NR_IRQS:64 nr_irqs:64 0
GICv3: GIC: Using split EOI/Deactivate mode
ITS [mem 0x801000020000-0x80100003ffff]
ITS@0x0000801000020000: allocated 2097152 Devices @1fc1000000 (flat, esz 
8, psz 64K, shr 1)
GIC: using LPI property table @0x0000001fc0060000
ITS: Allocated 32512 chunks for LPIs
GICv3: CPU0: found redistributor 0 region 0:0x0000801080000000
CPU0: using LPI pending table @0x0000001fc0070000
arm_arch_timer: Architected cp15 timer(s) running at 100.00MHz (phys).
clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 
0x171024e7e0, max_idle_ns: 440795205315 ns
sched_clock: 56 bits at 100MHz, resolution 10ns, wraps every 4398046511100ns
Console: colour dummy device 80x25
Calibrating delay loop (skipped), value calculated using timer 
frequency.. 200.00 BogoMIPS (lpj=400000)
pid_max: default: 49152 minimum: 384
ACPI: Core revision 20160831
ACPI: 2 ACPI AML tables successfully acquired and loaded
Security Framework initialized
Dentry cache hash table entries: 16777216 (order: 11, 134217728 bytes)
Inode-cache hash table entries: 8388608 (order: 10, 67108864 bytes)
Mount-cache hash table entries: 262144 (order: 5, 2097152 bytes)
Mountpoint-cache hash table entries: 262144 (order: 5, 2097152 bytes)
ASID allocator initialised with 65536 entries
PCI/MSI: ITS@0x801000020000 domain created
Remapping and enabling EFI services.
   EFI remap 0x0000001ffaa52000 => 0000000020002000
   EFI remap 0x0000001ffaa53000 => 0000000020003000
   EFI remap 0x0000001ffaa57000 => 0000000020007000
   EFI remap 0x0000001fffa42000 => 0000000020032000
   EFI remap 0x0000001fffa56000 => 0000000020046000
   EFI remap 0x0000804000001000 => 0000000020081000
   EFI remap 0x000087e0d2001000 => 0000000020091000
Detected VIPT I-cache on CPU1
GICv3: CPU1: found redistributor 1 region 0:0x0000801080020000
CPU1: using LPI pending table @0x0000001fcd880000
CPU1: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU2
GICv3: CPU2: found redistributor 2 region 0:0x0000801080040000
CPU2: using LPI pending table @0x0000001fcd8a0000
CPU2: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU3
GICv3: CPU3: found redistributor 3 region 0:0x0000801080060000
CPU3: using LPI pending table @0x0000001fcd8b0000
CPU3: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU4
GICv3: CPU4: found redistributor 4 region 0:0x0000801080080000
CPU4: using LPI pending table @0x0000001fcd8f0000
CPU4: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU5
GICv3: CPU5: found redistributor 5 region 0:0x00008010800a0000
CPU5: using LPI pending table @0x0000001fcd900000
CPU5: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU6
GICv3: CPU6: found redistributor 6 region 0:0x00008010800c0000
CPU6: using LPI pending table @0x0000001fcd930000
CPU6: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU7
GICv3: CPU7: found redistributor 7 region 0:0x00008010800e0000
CPU7: using LPI pending table @0x0000001fcd940000
CPU7: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU8
GICv3: CPU8: found redistributor 8 region 0:0x0000801080100000
CPU8: using LPI pending table @0x0000001fcd960000
CPU8: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU9
GICv3: CPU9: found redistributor 9 region 0:0x0000801080120000
CPU9: using LPI pending table @0x0000001fcda00000
CPU9: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU10
GICv3: CPU10: found redistributor a region 0:0x0000801080140000
CPU10: using LPI pending table @0x0000001fcda10000
CPU10: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU11
GICv3: CPU11: found redistributor b region 0:0x0000801080160000
CPU11: using LPI pending table @0x0000001fcda60000
CPU11: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU12
GICv3: CPU12: found redistributor c region 0:0x0000801080180000
CPU12: using LPI pending table @0x0000001fcda70000
CPU12: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU13
GICv3: CPU13: found redistributor d region 0:0x00008010801a0000
CPU13: using LPI pending table @0x0000001fcdb10000
CPU13: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU14
GICv3: CPU14: found redistributor e region 0:0x00008010801c0000
CPU14: using LPI pending table @0x0000001fcdb20000
CPU14: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU15
GICv3: CPU15: found redistributor f region 0:0x00008010801e0000
CPU15: using LPI pending table @0x0000001fcdb40000
CPU15: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU16
GICv3: CPU16: found redistributor 100 region 0:0x0000801080200000
CPU16: using LPI pending table @0x0000001fcdb60000
CPU16: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU17
GICv3: CPU17: found redistributor 101 region 0:0x0000801080220000
CPU17: using LPI pending table @0x0000001fcdc20000
CPU17: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU18
GICv3: CPU18: found redistributor 102 region 0:0x0000801080240000
CPU18: using LPI pending table @0x0000001fcdc30000
CPU18: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU19
GICv3: CPU19: found redistributor 103 region 0:0x0000801080260000
CPU19: using LPI pending table @0x0000001fcdc60000
CPU19: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU20
GICv3: CPU20: found redistributor 104 region 0:0x0000801080280000
CPU20: using LPI pending table @0x0000001fcdc70000
CPU20: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU21
GICv3: CPU21: found redistributor 105 region 0:0x00008010802a0000
CPU21: using LPI pending table @0x0000001fcdc90000
CPU21: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU22
GICv3: CPU22: found redistributor 106 region 0:0x00008010802c0000
CPU22: using LPI pending table @0x0000001fcdcb0000
CPU22: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU23
GICv3: CPU23: found redistributor 107 region 0:0x00008010802e0000
CPU23: using LPI pending table @0x0000001fcdcc0000
CPU23: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU24
GICv3: CPU24: found redistributor 108 region 0:0x0000801080300000
CPU24: using LPI pending table @0x0000001fcdd80000
CPU24: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU25
GICv3: CPU25: found redistributor 109 region 0:0x0000801080320000
CPU25: using LPI pending table @0x0000001fcdd90000
CPU25: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU26
GICv3: CPU26: found redistributor 10a region 0:0x0000801080340000
CPU26: using LPI pending table @0x0000001fcddc0000
CPU26: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU27
GICv3: CPU27: found redistributor 10b region 0:0x0000801080360000
CPU27: using LPI pending table @0x0000001fcddd0000
CPU27: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU28
GICv3: CPU28: found redistributor 10c region 0:0x0000801080380000
CPU28: using LPI pending table @0x0000001fcde80000
CPU28: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU29
GICv3: CPU29: found redistributor 10d region 0:0x00008010803a0000
CPU29: using LPI pending table @0x0000001fcde90000
CPU29: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU30
GICv3: CPU30: found redistributor 10e region 0:0x00008010803c0000
CPU30: using LPI pending table @0x0000001fcdeb0000
CPU30: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU31
GICv3: CPU31: found redistributor 10f region 0:0x00008010803e0000
CPU31: using LPI pending table @0x0000001fcdef0000
CPU31: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU32
GICv3: CPU32: found redistributor 200 region 0:0x0000801080400000
CPU32: using LPI pending table @0x0000001fcdf10000
CPU32: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU33
GICv3: CPU33: found redistributor 201 region 0:0x0000801080420000
CPU33: using LPI pending table @0x0000001fcdf40000
CPU33: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU34
GICv3: CPU34: found redistributor 202 region 0:0x0000801080440000
CPU34: using LPI pending table @0x0000001fcdf50000
CPU34: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU35
GICv3: CPU35: found redistributor 203 region 0:0x0000801080460000
CPU35: using LPI pending table @0x0000001fcdf70000
CPU35: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU36
GICv3: CPU36: found redistributor 204 region 0:0x0000801080480000
CPU36: using LPI pending table @0x0000001fce010000
CPU36: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU37
GICv3: CPU37: found redistributor 205 region 0:0x00008010804a0000
CPU37: using LPI pending table @0x0000001fce060000
CPU37: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU38
GICv3: CPU38: found redistributor 206 region 0:0x00008010804c0000
CPU38: using LPI pending table @0x0000001fce070000
CPU38: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU39
GICv3: CPU39: found redistributor 207 region 0:0x00008010804e0000
CPU39: using LPI pending table @0x0000001fce110000
CPU39: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU40
GICv3: CPU40: found redistributor 208 region 0:0x0000801080500000
CPU40: using LPI pending table @0x0000001fce120000
CPU40: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU41
GICv3: CPU41: found redistributor 209 region 0:0x0000801080520000
CPU41: using LPI pending table @0x0000001fce140000
CPU41: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU42
GICv3: CPU42: found redistributor 20a region 0:0x0000801080540000
CPU42: using LPI pending table @0x0000001fce160000
CPU42: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU43
GICv3: CPU43: found redistributor 20b region 0:0x0000801080560000
CPU43: using LPI pending table @0x0000001fce170000
CPU43: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU44
GICv3: CPU44: found redistributor 20c region 0:0x0000801080580000
CPU44: using LPI pending table @0x0000001fce230000
CPU44: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU45
GICv3: CPU45: found redistributor 20d region 0:0x00008010805a0000
CPU45: using LPI pending table @0x0000001fce240000
CPU45: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU46
GICv3: CPU46: found redistributor 20e region 0:0x00008010805c0000
CPU46: using LPI pending table @0x0000001fce270000
CPU46: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU47
GICv3: CPU47: found redistributor 20f region 0:0x00008010805e0000
CPU47: using LPI pending table @0x0000001fce290000
CPU47: Booted secondary processor [431f0a10]
Brought up 48 CPUs
SMP: Total of 48 processors activated.
CPU features: detected feature: GIC system register CPU interface
CPU features: detected feature: Software prefetching using PRFM
CPU: All CPU(s) started at EL2
alternatives: patching kernel code
devtmpfs: initialized
SMBIOS 3.0.0 present.
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, 
max_idle_ns: 7645041785100000 ns
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
cpuidle: using governor ladder
cpuidle: using governor menu
vdso: 2 pages (1 code @ ffff000008900000, 1 data @ ffff000008cb0000)
hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
DMA: preallocated 256 KiB pool for atomic allocations
ACPI: bus type PCI registered
Serial: AMBA PL011 UART driver
HugeTLB registered 2 MB page size, pre-allocated 0 pages
HugeTLB registered 512 MB page size, pre-allocated 0 pages
raid6: .... xor() 544 MB/s, rmw enabled
raid6: using intx1 recovery algorithm
ACPI: Added _OSI(Module Device)
ACPI: Added _OSI(Processor Device)
ACPI: Added _OSI(3.0 _SCP Extensions)
ACPI: Added _OSI(Processor Aggregator Device)
ACPI: Interpreter enabled
ACPI: Using GIC for interrupt routing
ACPI: MCFG table detected, 4 entries
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:00: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:00: ECAM area [mem 0x848000000000-0x848001ffffff] reserved 
by THRX0001:00
acpi PNP0A08:00: ECAM at [mem 0x848000000000-0x848001ffffff] for [bus 00-1f]
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x801000000000-0x807fffffffff 
window]
pci_bus 0000:00: root bus resource [mem 0x838000000000-0x841fffffffff 
window]
pci_bus 0000:00: root bus resource [mem 0x846000000000-0x847fffffffff 
window]
pci_bus 0000:00: root bus resource [mem 0x868000000000-0x87e023ffffff 
window]
pci_bus 0000:00: root bus resource [mem 0x87e026000000-0x87e0bfffffff 
window]
pci_bus 0000:00: root bus resource [mem 0x87e0c6000000-0x87ffffffffff 
window]
pci_bus 0000:00: root bus resource [bus 00-1f]
pci 0000:00:01.0: [177d:a002] type 01 class 0x060400
pci 0000:00:09.0: [177d:a018] type 00 class 0x120000
pci 0000:00:09.0: BAR 0: [mem 0x87e040000000-0x87e0400fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:00:09.0: VF BAR 0: [mem 0x840000800000-0x8400008fffff 64bit] 
(from Enhanced Allocation, properties 0x4)
pci 0000:00:09.0: VF(n) BAR0 space: [mem 0x840000800000-0x8400008fffff 
64bit] (contains BAR0 for 1 VFs)
pci 0000:00:10.0: [177d:a01b] type 00 class 0x0c0330
pci 0000:00:10.0: BAR 0: [mem 0x868000000000-0x8680001fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:00:10.0: BAR 4: [mem 0x868000200000-0x8680002fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:00:11.0: [177d:a01b] type 00 class 0x0c0330
pci 0000:00:11.0: BAR 0: [mem 0x869000000000-0x8690001fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:00:11.0: BAR 4: [mem 0x869000200000-0x8690002fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:00:14.0: [177d:a002] type 01 class 0x060400
pci 0000:00:15.0: [177d:a002] type 01 class 0x060400
pci 0000:00:16.0: [177d:a002] type 01 class 0x060400
pci 0000:01:00.0: [177d:a001] type 00 class 0x088000
pci 0000:01:00.0: BAR 0: [mem 0x87e0fc000000-0x87e0fc0fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:00.0: BAR 4: [mem 0x87e0fcf00000-0x87e0fcffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:00.1: [177d:a00e] type 00 class 0x088000
pci 0000:01:00.1: BAR 0: [mem 0x87e006000000-0x87e0067fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:00.1: BAR 4: [mem 0x87e006f00000-0x87e006ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:01.3: [177d:a02b] type 00 class 0x0c8000
pci 0000:01:01.3: BAR 0: [mem 0x87e005000000-0x87e0057fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:01.4: [177d:a010] type 00 class 0x058000
pci 0000:01:01.4: BAR 0: [mem 0x87e009000000-0x87e0097fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:01.4: BAR 4: [mem 0x87e009f00000-0x87e009ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.0: [177d:a02e] type 00 class 0x058000
pci 0000:01:06.0: BAR 0: [mem 0x87e050000000-0x87e0507fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.0: BAR 4: [mem 0x87e050f00000-0x87e050ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.1: [177d:a02e] type 00 class 0x058000
pci 0000:01:06.1: BAR 0: [mem 0x87e051000000-0x87e0517fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.1: BAR 4: [mem 0x87e051f00000-0x87e051ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.2: [177d:a02e] type 00 class 0x058000
pci 0000:01:06.2: BAR 0: [mem 0x87e052000000-0x87e0527fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.2: BAR 4: [mem 0x87e052f00000-0x87e052ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.3: [177d:a02e] type 00 class 0x058000
pci 0000:01:06.3: BAR 0: [mem 0x87e053000000-0x87e0537fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.3: BAR 4: [mem 0x87e053f00000-0x87e053ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.4: [177d:a02e] type 00 class 0x058000
pci 0000:01:06.4: BAR 0: [mem 0x87e054000000-0x87e0547fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.4: BAR 4: [mem 0x87e054f00000-0x87e054ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.5: [177d:a02e] type 00 class 0x058000
pci 0000:01:06.5: BAR 0: [mem 0x87e055000000-0x87e0557fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.5: BAR 4: [mem 0x87e055f00000-0x87e055ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.6: [177d:a02e] type 00 class 0x058000
pci 0000:01:06.6: BAR 0: [mem 0x87e056000000-0x87e0567fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.6: BAR 4: [mem 0x87e056f00000-0x87e056ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.7: [177d:a02e] type 00 class 0x058000
pci 0000:01:06.7: BAR 0: [mem 0x87e057000000-0x87e0577fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:06.7: BAR 4: [mem 0x87e057f00000-0x87e057ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.0: [177d:a02f] type 00 class 0x058000
pci 0000:01:07.0: BAR 0: [mem 0x87e058000000-0x87e0587fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.0: BAR 4: [mem 0x87e058f00000-0x87e058ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.1: [177d:a02f] type 00 class 0x058000
pci 0000:01:07.1: BAR 0: [mem 0x87e059000000-0x87e0597fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.1: BAR 4: [mem 0x87e059f00000-0x87e059ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.2: [177d:a02f] type 00 class 0x058000
pci 0000:01:07.2: BAR 0: [mem 0x87e05a000000-0x87e05a7fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.2: BAR 4: [mem 0x87e05af00000-0x87e05affffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.3: [177d:a02f] type 00 class 0x058000
pci 0000:01:07.3: BAR 0: [mem 0x87e05b000000-0x87e05b7fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.3: BAR 4: [mem 0x87e05bf00000-0x87e05bffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.4: [177d:a030] type 00 class 0x058000
pci 0000:01:07.4: BAR 0: [mem 0x87e05c000000-0x87e05c7fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.4: BAR 4: [mem 0x87e05cf00000-0x87e05cffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.5: [177d:a030] type 00 class 0x058000
pci 0000:01:07.5: BAR 0: [mem 0x87e05d000000-0x87e05d7fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.5: BAR 4: [mem 0x87e05df00000-0x87e05dffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.6: [177d:a030] type 00 class 0x058000
pci 0000:01:07.6: BAR 0: [mem 0x87e05e000000-0x87e05e7fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.6: BAR 4: [mem 0x87e05ef00000-0x87e05effffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.7: [177d:a030] type 00 class 0x058000
pci 0000:01:07.7: BAR 0: [mem 0x87e05f000000-0x87e05f7fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:07.7: BAR 4: [mem 0x87e05ff00000-0x87e05fffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:09.0: [177d:a012] type 00 class 0x0c8000
pci 0000:01:09.0: BAR 0: [mem 0x87e0d0000000-0x87e0d07fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:09.0: BAR 4: [mem 0x87e0d0f00000-0x87e0d0ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:09.1: [177d:a012] type 00 class 0x0c8000
pci 0000:01:09.1: BAR 0: [mem 0x87e0d1000000-0x87e0d17fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:09.1: BAR 4: [mem 0x87e0d1f00000-0x87e0d1ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:09.2: [177d:a012] type 00 class 0x0c8000
pci 0000:01:09.2: BAR 0: [mem 0x87e0d2000000-0x87e0d27fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:09.2: BAR 4: [mem 0x87e0d2f00000-0x87e0d2ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:09.3: [177d:a012] type 00 class 0x0c8000
pci 0000:01:09.3: BAR 0: [mem 0x87e0d3000000-0x87e0d37fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:09.3: BAR 4: [mem 0x87e0d3f00000-0x87e0d3ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:09.4: [177d:a012] type 00 class 0x0c8000
pci 0000:01:09.4: BAR 0: [mem 0x87e0d4000000-0x87e0d47fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:09.4: BAR 4: [mem 0x87e0d4f00000-0x87e0d4ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:0a.0: [177d:a022] type 00 class 0x058000
pci 0000:01:0a.0: BAR 0: [mem 0x87e088000000-0x87e0887fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:0a.0: BAR 4: [mem 0x87e088f00000-0x87e088ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:0a.1: [177d:a022] type 00 class 0x058000
pci 0000:01:0a.1: BAR 0: [mem 0x87e089000000-0x87e0897fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:0a.1: BAR 4: [mem 0x87e089f00000-0x87e089ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:0a.2: [177d:a022] type 00 class 0x058000
pci 0000:01:0a.2: BAR 0: [mem 0x87e08a000000-0x87e08a7fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:0a.2: BAR 4: [mem 0x87e08af00000-0x87e08affffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:0a.3: [177d:a022] type 00 class 0x058000
pci 0000:01:0a.3: BAR 0: [mem 0x87e08b000000-0x87e08b7fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:0a.3: BAR 4: [mem 0x87e08bf00000-0x87e08bffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:10.0: [177d:a026] type 00 class 0x028000
pci 0000:01:10.0: BAR 0: [mem 0x87e0e0000000-0x87e0e03fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:10.0: BAR 4: [mem 0x87e0e0400000-0x87e0e07fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:10.1: [177d:a026] type 00 class 0x028000
pci 0000:01:10.1: BAR 0: [mem 0x87e0e1000000-0x87e0e13fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:10.1: BAR 4: [mem 0x87e0e1400000-0x87e0e17fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:01:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0000:02:00.0: [177d:a01d] type 00 class 0x010400
pci 0000:02:00.0: BAR 0: [mem 0x870000000000-0x8700007fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:02:00.0: BAR 4: [mem 0x870000f00000-0x870000ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0000:03:00.0: [177d:a01a] type 00 class 0x120000
pci 0000:03:00.0: BAR 0: [mem 0x838000000000-0x8380003fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:03:00.0: BAR 4: [mem 0x838000f00000-0x838000ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0000:04:00.0: [177d:a019] type 00 class 0x120000
pci 0000:04:00.0: BAR 0: [mem 0x846000000000-0x8467ffffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:04:00.0: BAR 4: [mem 0x846a00000000-0x846a000fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0000:04:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0000:00:01.0: PCI bridge to [bus 01]
pci 0000:00:14.0: PCI bridge to [bus 02]
pci 0000:00:15.0: PCI bridge to [bus 03]
pci 0000:00:16.0: PCI bridge to [bus 04]
ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus 00-1f])
acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:01: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:01: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:01: ECAM area [mem 0x849000000000-0x849001ffffff] reserved 
by THRX0001:01
acpi PNP0A08:01: ECAM at [mem 0x849000000000-0x849001ffffff] for [bus 00-1f]
PCI host bridge to bus 0001:00
pci_bus 0001:00: root bus resource [mem 0x810000000000-0x817fffffffff 
window]
pci_bus 0001:00: root bus resource [bus 00-1f]
pci 0001:00:08.0: [177d:a01c] type 00 class 0x010601
pci 0001:00:08.0: BAR 0: [mem 0x814000000000-0x8140001fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0001:00:08.0: BAR 4: [mem 0x814000200000-0x8140002fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0001:00:09.0: [177d:a01c] type 00 class 0x010601
pci 0001:00:09.0: BAR 0: [mem 0x815000000000-0x8150001fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0001:00:09.0: BAR 4: [mem 0x815000200000-0x8150002fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0001:00:0a.0: [177d:a01c] type 00 class 0x010601
pci 0001:00:0a.0: BAR 0: [mem 0x816000000000-0x8160001fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0001:00:0a.0: BAR 4: [mem 0x816000200000-0x8160002fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0001:00:0b.0: [177d:a01c] type 00 class 0x010601
pci 0001:00:0b.0: BAR 0: [mem 0x817000000000-0x8170001fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0001:00:0b.0: BAR 4: [mem 0x817000200000-0x8170002fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 00-1f])
acpi PNP0A08:02: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:02: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:02: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:02: ECAM area [mem 0x84a000000000-0x84a001ffffff] reserved 
by THRX0001:02
acpi PNP0A08:02: ECAM at [mem 0x84a000000000-0x84a001ffffff] for [bus 00-1f]
PCI host bridge to bus 0002:00
pci_bus 0002:00: root bus resource [mem 0x842000000000-0x843fffffffff 
window]
pci_bus 0002:00: root bus resource [bus 00-1f]
pci 0002:00:02.0: [177d:a002] type 01 class 0x060400
pci 0002:00:03.0: [177d:a01f] type 00 class 0x028000
pci 0002:00:03.0: BAR 0: [mem 0x842000000000-0x84200000ffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0002:00:03.0: BAR 2: [mem 0x842040000000-0x84207fffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0002:00:03.0: BAR 4: [mem 0x842000f00000-0x842000ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0002:01:00.0: [177d:a01e] type 00 class 0x020000
pci 0002:01:00.0: BAR 0: [mem 0x843000000000-0x84303fffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0002:01:00.0: BAR 4: [mem 0x843060000000-0x8430600fffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0002:01:00.0: VF BAR 0: [mem 0x8430a0000000-0x8430a01fffff 64bit] 
(from Enhanced Allocation, properties 0x4)
pci 0002:01:00.0: VF BAR 4: [mem 0x8430e0000000-0x8430e01fffff 64bit] 
(from Enhanced Allocation, properties 0x4)
pci 0002:01:00.0: VF(n) BAR0 space: [mem 0x8430a0000000-0x8430afffffff 
64bit] (contains BAR0 for 128 VFs)
pci 0002:01:00.0: VF(n) BAR4 space: [mem 0x8430e0000000-0x8430efffffff 
64bit] (contains BAR4 for 128 VFs)
pci 0002:01:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0002:00:02.0: PCI bridge to [bus 01]
ACPI: PCI Root Bridge [PCI3] (domain 0003 [bus 00-1f])
acpi PNP0A08:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:03: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:03: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:03: ECAM area [mem 0x84b000000000-0x84b001ffffff] reserved 
by THRX0001:03
acpi PNP0A08:03: ECAM at [mem 0x84b000000000-0x84b001ffffff] for [bus 00-1f]
PCI host bridge to bus 0003:00
pci_bus 0003:00: root bus resource [mem 0x818000000000-0x81ffffffffff 
window]
pci_bus 0003:00: root bus resource [bus 00-1f]
ACPI: PCI Root Bridge [PEM0] (domain 0004 [bus 1f-57])
acpi PNP0A08:04: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:04: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:04: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:04: MCFG quirk: ECAM at [mem 0x88001f000000-0x880057ffffff] 
for [bus 1f-57] with thunder_pem_ecam_ops
acpi PNP0A08:04: ECAM area [mem 0x88001f000000-0x880057ffffff] reserved 
by THRX0002:00
acpi PNP0A08:04: ECAM at [mem 0x88001f000000-0x880057ffffff] for [bus 1f-57]
Remapped I/O 0x0000883000000000 to [io  0x0000-0xffff window]
PCI host bridge to bus 0004:1f
pci_bus 0004:1f: root bus resource [io  0x0000-0xffff window]
pci_bus 0004:1f: root bus resource [mem 0x881010000000-0x881fffffffff 
window] (bus address [0x10000000-0xfffffffff])
pci_bus 0004:1f: root bus resource [mem 0x882000000000-0x882fffffffff 
pref window] (bus address [0x1000000000-0x1fffffffff])
pci_bus 0004:1f: root bus resource [mem 0x87e0c0000000-0x87e0c0ffffff 
window]
pci_bus 0004:1f: root bus resource [bus 1f-57]
pci 0004:1f:00.0: [177d:a100] type 01 class 0x060400
pci 0004:1f:00.0: reg 0x10: [mem 0x00000000-0x00007fff 64bit pref]
pci 0004:1f:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
pci 0004:1f:00.0: BAR 0: [mem 0x87e0c0f00000-0x87e0c0ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0004:1f:00.0: Primary bus is hard wired to 0
pci 0004:20:00.0: [1a03:1150] type 01 class 0x060400
pci 0004:20:00.0: can't set Max Payload Size to 256; if necessary, use 
"pci=pcie_bus_safe" and report a bug
pci 0004:20:00.0: supports D1 D2
pci 0004:20:00.0: PME# supported from D0 D1 D2 D3hot D3cold
pci 0004:20:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0004:21:00.0: [1a03:2000] type 00 class 0x030000
pci 0004:21:00.0: reg 0x10: [mem 0x881010000000-0x881010ffffff]
pci 0004:21:00.0: reg 0x14: [mem 0x881011000000-0x88101101ffff]
pci 0004:21:00.0: reg 0x18: [io  0x0000-0x007f]
pci 0004:21:00.0: supports D1 D2
pci 0004:21:00.0: PME# supported from D0 D1 D2 D3hot D3cold
pci 0004:1f:00.0: Primary bus is hard wired to 0
pci 0004:1f:00.0: BAR 14: assigned [mem 0x881010000000-0x8810117fffff]
pci 0004:1f:00.0: BAR 6: assigned [mem 0x881011800000-0x88101180ffff pref]
pci 0004:1f:00.0: BAR 13: assigned [io  0x1000-0x1fff]
pci 0004:20:00.0: BAR 14: assigned [mem 0x881010000000-0x8810117fffff]
pci 0004:20:00.0: BAR 13: assigned [io  0x1000-0x1fff]
pci 0004:21:00.0: BAR 0: assigned [mem 0x881010000000-0x881010ffffff]
pci 0004:21:00.0: BAR 1: assigned [mem 0x881011000000-0x88101101ffff]
pci 0004:21:00.0: BAR 2: assigned [io  0x1000-0x107f]
pci 0004:20:00.0: PCI bridge to [bus 21]
pci 0004:20:00.0:   bridge window [io  0x1000-0x1fff]
pci 0004:20:00.0:   bridge window [mem 0x881010000000-0x8810117fffff]
pci 0004:1f:00.0: PCI bridge to [bus 20-57]
pci 0004:1f:00.0:   bridge window [io  0x1000-0x1fff]
pci 0004:1f:00.0:   bridge window [mem 0x881010000000-0x8810117fffff]
ACPI: PCI Interrupt Link [LN0A] (IRQs *48)
ACPI: PCI Interrupt Link [LN0B] (IRQs *49)
ACPI: PCI Interrupt Link [LN0C] (IRQs *50)
ACPI: PCI Interrupt Link [LN0D] (IRQs *51)
ACPI: PCI Root Bridge [PEM2] (domain 0006 [bus 8f-c7])
acpi PNP0A08:06: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:06: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:06: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:06: MCFG quirk: ECAM at [mem 0x88808f000000-0x8880c7ffffff] 
for [bus 8f-c7] with thunder_pem_ecam_ops
acpi PNP0A08:06: ECAM area [mem 0x88808f000000-0x8880c7ffffff] reserved 
by THRX0002:02
acpi PNP0A08:06: ECAM at [mem 0x88808f000000-0x8880c7ffffff] for [bus 8f-c7]
Remapped I/O 0x000088b000020000 to [io  0x10000-0x1ffff window]
PCI host bridge to bus 0006:8f
pci_bus 0006:8f: root bus resource [io  0x10000-0x1ffff window] (bus 
address [0x20000-0x2ffff])
pci_bus 0006:8f: root bus resource [mem 0x889010000000-0x889fffffffff 
window] (bus address [0x10000000-0xfffffffff])
pci_bus 0006:8f: root bus resource [mem 0x88a000000000-0x88afffffffff 
pref window] (bus address [0x1000000000-0x1fffffffff])
pci_bus 0006:8f: root bus resource [mem 0x87e0c2000000-0x87e0c2ffffff 
window]
pci_bus 0006:8f: root bus resource [bus 8f-c7]
pci 0006:8f:00.0: [177d:a100] type 01 class 0x060400
pci 0006:8f:00.0: reg 0x10: [mem 0x00000000-0x00007fff 64bit pref]
pci 0006:8f:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
pci 0006:8f:00.0: BAR 0: [mem 0x87e0c2f00000-0x87e0c2ffffff 64bit] (from 
Enhanced Allocation, properties 0x0)
pci 0006:8f:00.0: Primary bus is hard wired to 0
pci 0006:90:00.0: [8086:10fb] type 00 class 0x020000
pci 0006:90:00.0: reg 0x10: [mem 0x889010000000-0x88901007ffff 64bit pref]
pci 0006:90:00.0: reg 0x18: [io  0x0000-0x001f]
pci 0006:90:00.0: reg 0x20: [mem 0x889010080000-0x889010083fff 64bit pref]
pci 0006:90:00.0: Max Payload Size set to 256 (was 128, max 512)
pci 0006:90:00.0: PME# supported from D0 D3hot
pci 0006:90:00.0: reg 0x184: [mem 0x00000000-0x0000ffff 64bit]
pci 0006:90:00.0: VF(n) BAR0 space: [mem 0x00000000-0x003fffff 64bit] 
(contains BAR0 for 64 VFs)
pci 0006:90:00.0: reg 0x190: [mem 0x00000000-0x0000ffff 64bit]
pci 0006:90:00.0: VF(n) BAR3 space: [mem 0x00000000-0x003fffff 64bit] 
(contains BAR3 for 64 VFs)
pci 0006:8f:00.0: Primary bus is hard wired to 0
pci 0006:8f:00.0: BAR 14: assigned [mem 0x889010000000-0x8890107fffff]
pci 0006:8f:00.0: BAR 15: assigned [mem 0x88a000000000-0x88a0000fffff 
64bit pref]
pci 0006:8f:00.0: BAR 6: assigned [mem 0x889010800000-0x88901080ffff pref]
pci 0006:8f:00.0: BAR 13: assigned [io  0x10000-0x10fff]
pci 0006:90:00.0: BAR 0: assigned [mem 0x88a000000000-0x88a00007ffff 
64bit pref]
pci 0006:90:00.0: BAR 7: assigned [mem 0x889010000000-0x8890103fffff 64bit]
pci 0006:90:00.0: BAR 10: assigned [mem 0x889010400000-0x8890107fffff 64bit]
pci 0006:90:00.0: BAR 4: assigned [mem 0x88a000080000-0x88a000083fff 
64bit pref]
pci 0006:90:00.0: BAR 2: assigned [io  0x10000-0x1001f]
pci 0006:8f:00.0: PCI bridge to [bus 90-c7]
pci 0006:8f:00.0:   bridge window [io  0x10000-0x10fff]
pci 0006:8f:00.0:   bridge window [mem 0x889010000000-0x8890107fffff]
pci 0006:8f:00.0:   bridge window [mem 0x88a000000000-0x88a0000fffff 
64bit pref]
ACPI: PCI Interrupt Link [LN0A] (IRQs *56)
ACPI: PCI Interrupt Link [LN0B] (IRQs *57)
ACPI: PCI Interrupt Link [LN0C] (IRQs *58)
ACPI: PCI Interrupt Link [LN0D] (IRQs *59)
ARMH0011:00: ttyAMA0 at MMIO 0x87e024000000 (irq = 10, base_baud = 0) is 
a SBSA
console [ttyAMA0] enabled
console [ttyAMA0] enabled
bootconsole [pl11] disabled
bootconsole [pl11] disabled
ARMH0011:01: ttyAMA1 at MMIO 0x87e025000000 (irq = 11, base_baud = 0) is 
a SBSA
vgaarb: device added: PCI:0004:21:00.0,decodes=io+mem,owns=none,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0004:21:00.0
SCSI subsystem initialized
libata version 3.00 loaded.
ACPI: bus type USB registered
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti 
<giometti@linux.it>
PTP clock support registered
Registered efivars operations
clocksource: Switched to clocksource arch_sys_counter
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
pnp: PnP ACPI init
system 00:00: [mem 0x848000000000-0x848001ffffff] could not be reserved
system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active)
system 00:01: [mem 0x849000000000-0x849001ffffff] could not be reserved
system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
system 00:02: [mem 0x84a000000000-0x84a001ffffff] could not be reserved
system 00:02: Plug and Play ACPI device, IDs PNP0c02 (active)
system 00:03: [mem 0x84b000000000-0x84b001ffffff] could not be reserved
system 00:03: Plug and Play ACPI device, IDs PNP0c02 (active)
system 00:04: [mem 0x87e0c0000000-0x87e0c0ffffff] could not be reserved
system 00:04: [mem 0x88001f000000-0x880057ffffff] could not be reserved
system 00:04: Plug and Play ACPI device, IDs PNP0c02 (active)
system 00:05: [mem 0x87e0c2000000-0x87e0c2ffffff] could not be reserved
system 00:05: [mem 0x88808f000000-0x8880c7ffffff] could not be reserved
system 00:05: Plug and Play ACPI device, IDs PNP0c02 (active)
pnp: PnP ACPI: found 6 devices
NET: Registered protocol family 2
TCP established hash table entries: 524288 (order: 6, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 4, 1048576 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
UDP hash table entries: 65536 (order: 5, 2097152 bytes)
UDP-Lite hash table entries: 65536 (order: 5, 2097152 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
PCI: CLS 64 bytes, default 128
kvm [1]: 16-bit VMID
kvm [1]: IDMAP page: 1ccb000
kvm [1]: HYP VA range: 800000000000:ffffffffffff
kvm [1]: Hyp mode initialized successfully
kvm [1]: GICv3: no GICV resource entry
kvm [1]: disabling GICv2 emulation
kvm [1]: GIC system register CPU interface enabled
kvm [1]: vgic interrupt IRQ1
kvm [1]: virtual timer IRQ4
futex hash table entries: 16384 (order: 5, 2097152 bytes)
audit: initializing netlink subsys (disabled)
audit: type=2000 audit(8.064:1): initialized
workingset: timestamp_bits=44 max_order=21 bucket_order=0
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
nfs4filelayout_init: NFSv4 File Layout Driver Registering...
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
fuse init (API version 7.26)
9p: Installing v9fs 9p2000 file system support
io scheduler noop registered
io scheduler cfq registered (default)
ACPI: PCI Interrupt Link [LN0A] enabled at IRQ 48
pcieport 0004:1f:00.0: enabling device (0506 -> 0507)
ACPI: PCI Interrupt Link [LN0A] enabled at IRQ 56
pcieport 0006:8f:00.0: enabling device (0506 -> 0507)
acpi-ged ACPI0013:00: GED listening GSI 32 @ IRQ 12
input: Power Button as 
/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
ACPI: Power Button [PWRB]
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
msm_serial: driver initialized
Failed to find cpu0 device node
Unable to detect cache hierarchy from DT for CPU 0
loop: module loaded
ahci 0001:00:08.0: version 3.0
ahci 0001:00:08.0: AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
ahci 0001:00:08.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs 
pio slum part ccc apst
ahci 0001:00:08.0: port 0 is not capable of FBS
scsi host0: ahci
ata1: SATA max UDMA/133 abar m2097152@0x814000000000 port 0x814000000100 
irq 14
ahci 0001:00:09.0: AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
ahci 0001:00:09.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs 
pio slum part ccc apst
ahci 0001:00:09.0: port 0 is not capable of FBS
scsi host1: ahci
ata2: SATA max UDMA/133 abar m2097152@0x815000000000 port 0x815000000100 
irq 15
ahci 0001:00:0a.0: AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
ahci 0001:00:0a.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs 
pio slum part ccc apst
ahci 0001:00:0a.0: port 0 is not capable of FBS
scsi host2: ahci
ata3: SATA max UDMA/133 abar m2097152@0x816000000000 port 0x816000000100 
irq 16
ahci 0001:00:0b.0: AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
ahci 0001:00:0b.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs 
pio slum part ccc apst
ahci 0001:00:0b.0: port 0 is not capable of FBS
scsi host3: ahci
ata4: SATA max UDMA/133 abar m2097152@0x817000000000 port 0x817000000100 
irq 17
libphy: Fixed MDIO Bus: probed
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
thunder-xcv, ver 1.0
thunder-BGX, ver 1.0
thunder-BGX 0000:01:10.0: BGX0 QLM mode: XLAUI
thunder-BGX 0000:01:10.0: MAC address set to: 40:8d:5c:ba:b8:ea
thunder-BGX 0000:01:10.1: BGX1 QLM mode: XFI
thunder-BGX 0000:01:10.1: MAC address set to: 40:8d:5c:ba:b8:eb
thunder-BGX 0000:01:10.1: MAC address set to: 40:8d:5c:ba:b8:ec
thunder-nic, ver 1.0
pci 0002:01:00.1: [177d:a034] type 00 class 0x020000
pci 0002:01:00.2: [177d:a034] type 00 class 0x020000
pci 0002:01:00.3: [177d:a034] type 00 class 0x020000
pci 0002:01:00.4: [177d:a034] type 00 class 0x020000
pci 0002:01:00.5: [177d:a034] type 00 class 0x020000
pci 0002:01:00.6: [177d:a034] type 00 class 0x020000
pci 0002:01:00.7: [177d:a034] type 00 class 0x020000
pci 0002:01:01.0: [177d:a034] type 00 class 0x020000
pci 0002:01:01.1: [177d:a034] type 00 class 0x020000
pci 0002:01:01.2: [177d:a034] type 00 class 0x020000
pci 0002:01:01.3: [177d:a034] type 00 class 0x020000
pci 0002:01:01.4: [177d:a034] type 00 class 0x020000
pci 0002:01:01.5: [177d:a034] type 00 class 0x020000
pci 0002:01:01.6: [177d:a034] type 00 class 0x020000
pci 0002:01:01.7: [177d:a034] type 00 class 0x020000
pci 0002:01:02.0: [177d:a034] type 00 class 0x020000
pci 0002:01:02.1: [177d:a034] type 00 class 0x020000
pci 0002:01:02.2: [177d:a034] type 00 class 0x020000
pci 0002:01:02.3: [177d:a034] type 00 class 0x020000
pci 0002:01:02.4: [177d:a034] type 00 class 0x020000
pci 0002:01:02.5: [177d:a034] type 00 class 0x020000
pci 0002:01:02.6: [177d:a034] type 00 class 0x020000
pci 0002:01:02.7: [177d:a034] type 00 class 0x020000
pci 0002:01:03.0: [177d:a034] type 00 class 0x020000
pci 0002:01:03.1: [177d:a034] type 00 class 0x020000
pci 0002:01:03.2: [177d:a034] type 00 class 0x020000
pci 0002:01:03.3: [177d:a034] type 00 class 0x020000
pci 0002:01:03.4: [177d:a034] type 00 class 0x020000
pci 0002:01:03.5: [177d:a034] type 00 class 0x020000
pci 0002:01:03.6: [177d:a034] type 00 class 0x020000
pci 0002:01:03.7: [177d:a034] type 00 class 0x020000
pci 0002:01:04.0: [177d:a034] type 00 class 0x020000
pci 0002:01:04.1: [177d:a034] type 00 class 0x020000
pci 0002:01:04.2: [177d:a034] type 00 class 0x020000
pci 0002:01:04.3: [177d:a034] type 00 class 0x020000
pci 0002:01:04.4: [177d:a034] type 00 class 0x020000
thunder-nic 0002:01:00.0: SRIOV enabled, number of VF available 36
thunder-nicvf, ver 1.0
thunder-nicvf 0002:01:00.1: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:00.2: enabling device (0004 -> 0006)
ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
ata1.00: ATA-8: WDC WD5003ABYZ-011FA0, 01.01S03, max UDMA/133
ata1.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
ata1.00: configured for UDMA/133
scsi 0:0:0:0: Direct-Access     ATA      WDC WD5003ABYZ-0 1S03 PQ: 0 ANSI: 5
ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
ata2.00: ATA-9: WDC WD5000AZLX-00CL5A0, 01.01A01, max UDMA/133
ata2.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
ata2.00: configured for UDMA/133
thunder-nicvf 0002:01:00.3: enabling device (0004 -> 0006)
ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
ata3.00: ATA-8: WDC WD5003ABYZ-011FA0, 01.01S03, max UDMA/133
ata3.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
thunder-nicvf 0002:01:00.4: enabling device (0004 -> 0006)
ata3.00: configured for UDMA/133
ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/466 GiB)
scsi 1:0:0:0: Direct-Access     ATA      WDC WD5000AZLX-0 1A01 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] Write Protect is off
ata4.00: ATA-8: OCZ-VERTEX3, 2.25, max UDMA/133
thunder-nicvf 0002:01:00.5: enabling device (0004 -> 0006)
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
ata4.00: 234441648 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
sd 1:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/466 GiB)
scsi 2:0:0:0: Direct-Access     ATA      WDC WD5003ABYZ-0 1S03 PQ: 0 ANSI: 5
ata4.00: configured for UDMA/133
  sda: sda1 sda2 sda3
sd 0:0:0:0: [sda] Attached SCSI disk
thunder-nicvf 0002:01:00.6: enabling device (0004 -> 0006)
sd 1:0:0:0: [sdb] 4096-byte physical blocks
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
thunder-nicvf 0002:01:00.7: enabling device (0004 -> 0006)
sd 2:0:0:0: [sdc] 976773168 512-byte logical blocks: (500 GB/466 GiB)
scsi 3:0:0:0: Direct-Access     ATA      OCZ-VERTEX3      2.25 PQ: 0 ANSI: 5
sd 1:0:0:0: [sdb] Attached SCSI disk
sd 2:0:0:0: [sdc] Write Protect is off
sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00
sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
thunder-nicvf 0002:01:01.0: enabling device (0004 -> 0006)
  sdc: sdc1 sdc2 sdc3
sd 2:0:0:0: [sdc] Attached SCSI disk
thunder-nicvf 0002:01:01.1: enabling device (0004 -> 0006)
sd 3:0:0:0: [sdd] 234441648 512-byte logical blocks: (120 GB/112 GiB)
sd 3:0:0:0: [sdd] Write Protect is off
sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
  sdd: sdd1
sd 3:0:0:0: [sdd] Attached SCSI disk
thunder-nicvf 0002:01:01.2: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:01.3: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:01.4: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:01.5: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:01.6: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:01.7: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.0: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.1: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.2: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.3: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.4: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.5: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.6: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.7: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.0: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.1: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.2: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.3: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.4: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.5: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.6: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.7: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:04.0: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:04.1: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:04.2: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:04.3: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:04.4: enabling device (0004 -> 0006)
dmfe: Davicom DM9xxx net driver, version 1.36.4 (2002-01-17)
v1.01-e (2.4 port) Sep-11-2006  Donald Becker <becker@scyld.com>
   http://www.scyld.com/network/drivers.html
uli526x: ULi M5261/M5263 net driver, version 0.9.3 (2005-7-29)
e100: Intel(R) PRO/100 Network Driver, 3.5.24-k2-NAPI
e100: Copyright(c) 1999-2006 Intel Corporation
e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
e1000: Copyright (c) 1999-2006 Intel Corporation.
e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
igb: Copyright (c) 2007-2014 Intel Corporation.
igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 4.4.0-k
ixgbe: Copyright (c) 1999-2016 Intel Corporation.
ixgbe 0006:90:00.0: enabling device (0000 -> 0002)
ixgbe 0006:90:00.0: Multiqueue Enabled: Rx Queue count = 48, Tx Queue 
count = 48
ixgbe 0006:90:00.0: PCI Express bandwidth of 32GT/s available
ixgbe 0006:90:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
ixgbe 0006:90:00.0: MAC: 2, PHY: 18, SFP+: 5, PBA No: E68787-006
ixgbe 0006:90:00.0: 90:e2:ba:16:d1:86
ixgbe 0006:90:00.0: Intel(R) 10 Gigabit Network Connection
ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver 
- version 3.2.2-k
ixgbevf: Copyright (c) 2009 - 2015 Intel Corporation.
i40e: Intel(R) Ethernet Connection XL710 Network Driver - version 1.6.16-k
i40e: Copyright (c) 2013 - 2014 Intel Corporation.
ixgb: Intel(R) PRO/10GbE Network Driver - version 1.0.135-k2-NAPI
ixgb: Copyright (c) 1999-2008 Intel Corporation.
i40evf: Intel(R) 40-10 Gigabit Virtual Function Network Driver - version 
1.6.16-k
Copyright (c) 2013 - 2015 Intel Corporation.
Intel(R) Ethernet Switch Host Interface Driver - version 0.21.2-k
Copyright (c) 2013 - 2016 Intel Corporation.
sky2: driver version 1.30
usbcore: registered new interface driver asix
usbcore: registered new interface driver ax88179_178a
usbcore: registered new interface driver cdc_ether
usbcore: registered new interface driver net1080
usbcore: registered new interface driver cdc_subset
usbcore: registered new interface driver zaurus
usbcore: registered new interface driver cdc_ncm
VFIO - User Level meta-driver version: 0.3
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
ehci-platform: EHCI generic platform driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-pci: OHCI PCI platform driver
ohci-platform: OHCI generic platform driver
uhci_hcd: USB Universal Host Controller Interface driver
xhci_hcd 0000:00:10.0: xHCI Host Controller
xhci_hcd 0000:00:10.0: new USB bus registered, assigned bus number 1
xhci_hcd 0000:00:10.0: hcc params 0x0220f665 hci version 0x100 quirks 
0x00000010
xhci_hcd 0000:00:10.0: cache line size of 64 is not supported
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 1 port detected
xhci_hcd 0000:00:10.0: xHCI Host Controller
xhci_hcd 0000:00:10.0: new USB bus registered, assigned bus number 2
usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 1 port detected
xhci_hcd 0000:00:11.0: xHCI Host Controller
xhci_hcd 0000:00:11.0: new USB bus registered, assigned bus number 3
xhci_hcd 0000:00:11.0: hcc params 0x0220f665 hci version 0x100 quirks 
0x00000010
xhci_hcd 0000:00:11.0: cache line size of 64 is not supported
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 1 port detected
xhci_hcd 0000:00:11.0: xHCI Host Controller
xhci_hcd 0000:00:11.0: new USB bus registered, assigned bus number 4
usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 1 port detected
usbcore: registered new interface driver usb-storage
mousedev: PS/2 mouse device common for all mice
device-mapper: ioctl: 4.35.0-ioctl (2016-06-23) initialised: 
dm-devel@redhat.com
sdhci: Secure Digital Host Controller Interface driver
sdhci: Copyright(c) Pierre Ossman
sdhci-pltfm: SDHCI platform and OF driver helper
ledtrig-cpu: registered to indicate activity on CPUs
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
Netfilter messages via NETLINK v0.30.
nfnl_acct: registering with nfnetlink.
nf_conntrack version 0.5.0 (65536 buckets, 262144 max)
ip_set: protocol 6
IPVS: Registered protocols ()
IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
IPVS: Creating netns size=1336 id=0
IPVS: ipvs loaded.
ip_tables: (C) 2000-2006 Netfilter Core Team
arp_tables: arp_tables: (C) 2002 David S. Miller
NET: Registered protocol family 17
bridge: filtering via arp/ip/ip6tables is no longer available by 
default. Update your scripts to load br_netfilter if you need this.

9pnet: Installing 9P2000 support
Key type dns_resolver registered
registered taskstats version 1
usb 3-1: new high-speed USB device number 2 using xhci_hcd
hctosys: unable to open rtc device (rtc0)
EXT4-fs (sda2): couldn't mount as ext3 due to feature incompatibilities
EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext4 filesystem) on device 8:2.
devtmpfs: mounted
Freeing unused kernel memory: 640K (ffff800002000000 - ffff8000020a0000)
random: fast init done

# cat /proc/iomem
01400000-fffeffff : System RAM
   01480000-01ffffff : Kernel code
   020a0000-0225ffff : Kernel data
ffff0000-ffffffff : reserved
100000000-1ffaa4ffff : System RAM
1ffaa50000-1ffaa7ffff : reserved
1ffaa80000-1fffa3ffff : System RAM
1fffa40000-1fffa8ffff : reserved
1fffa90000-1fffffffff : System RAM
801000000000-807fffffffff : PCI Bus 0000:00
810000000000-817fffffffff : PCI Bus 0001:00
   814000000000-8140001fffff : 0001:00:08.0
     814000000000-8140001fffff : ahci
   814000200000-8140002fffff : 0001:00:08.0
     814000200000-8140002fffff : ahci
   815000000000-8150001fffff : 0001:00:09.0
     815000000000-8150001fffff : ahci
   815000200000-8150002fffff : 0001:00:09.0
     815000200000-8150002fffff : ahci
   816000000000-8160001fffff : 0001:00:0a.0
     816000000000-8160001fffff : ahci
   816000200000-8160002fffff : 0001:00:0a.0
     816000200000-8160002fffff : ahci
   817000000000-8170001fffff : 0001:00:0b.0
     817000000000-8170001fffff : ahci
   817000200000-8170002fffff : 0001:00:0b.0
     817000200000-8170002fffff : ahci
818000000000-81ffffffffff : PCI Bus 0003:00
838000000000-841fffffffff : PCI Bus 0000:00
   838000000000-8380003fffff : 0000:03:00.0
   838000f00000-838000ffffff : 0000:03:00.0
   840000800000-8400008fffff : 0000:00:09.0
842000000000-843fffffffff : PCI Bus 0002:00
   842000000000-84200000ffff : 0002:00:03.0
   842000f00000-842000ffffff : 0002:00:03.0
   842040000000-84207fffffff : 0002:00:03.0
   843000000000-84303fffffff : 0002:01:00.0
     843000000000-84303fffffff : thunder-nic
   843060000000-8430600fffff : 0002:01:00.0
     843060000000-8430600fffff : thunder-nic
   8430a0000000-8430afffffff : 0002:01:00.0
     8430a0000000-8430a01fffff : 0002:01:00.1
       8430a0000000-8430a01fffff : thunder-nicvf
     8430a0200000-8430a03fffff : 0002:01:00.2
       8430a0200000-8430a03fffff : thunder-nicvf
     8430a0400000-8430a05fffff : 0002:01:00.3
       8430a0400000-8430a05fffff : thunder-nicvf
     8430a0600000-8430a07fffff : 0002:01:00.4
       8430a0600000-8430a07fffff : thunder-nicvf
     8430a0800000-8430a09fffff : 0002:01:00.5
       8430a0800000-8430a09fffff : thunder-nicvf
     8430a0a00000-8430a0bfffff : 0002:01:00.6
       8430a0a00000-8430a0bfffff : thunder-nicvf
     8430a0c00000-8430a0dfffff : 0002:01:00.7
       8430a0c00000-8430a0dfffff : thunder-nicvf
     8430a0e00000-8430a0ffffff : 0002:01:01.0
       8430a0e00000-8430a0ffffff : thunder-nicvf
     8430a1000000-8430a11fffff : 0002:01:01.1
       8430a1000000-8430a11fffff : thunder-nicvf
     8430a1200000-8430a13fffff : 0002:01:01.2
       8430a1200000-8430a13fffff : thunder-nicvf
     8430a1400000-8430a15fffff : 0002:01:01.3
       8430a1400000-8430a15fffff : thunder-nicvf
     8430a1600000-8430a17fffff : 0002:01:01.4
       8430a1600000-8430a17fffff : thunder-nicvf
     8430a1800000-8430a19fffff : 0002:01:01.5
       8430a1800000-8430a19fffff : thunder-nicvf
     8430a1a00000-8430a1bfffff : 0002:01:01.6
       8430a1a00000-8430a1bfffff : thunder-nicvf
     8430a1c00000-8430a1dfffff : 0002:01:01.7
       8430a1c00000-8430a1dfffff : thunder-nicvf
     8430a1e00000-8430a1ffffff : 0002:01:02.0
       8430a1e00000-8430a1ffffff : thunder-nicvf
     8430a2000000-8430a21fffff : 0002:01:02.1
       8430a2000000-8430a21fffff : thunder-nicvf
     8430a2200000-8430a23fffff : 0002:01:02.2
       8430a2200000-8430a23fffff : thunder-nicvf
     8430a2400000-8430a25fffff : 0002:01:02.3
       8430a2400000-8430a25fffff : thunder-nicvf
     8430a2600000-8430a27fffff : 0002:01:02.4
       8430a2600000-8430a27fffff : thunder-nicvf
     8430a2800000-8430a29fffff : 0002:01:02.5
       8430a2800000-8430a29fffff : thunder-nicvf
     8430a2a00000-8430a2bfffff : 0002:01:02.6
       8430a2a00000-8430a2bfffff : thunder-nicvf
     8430a2c00000-8430a2dfffff : 0002:01:02.7
       8430a2c00000-8430a2dfffff : thunder-nicvf
     8430a2e00000-8430a2ffffff : 0002:01:03.0
       8430a2e00000-8430a2ffffff : thunder-nicvf
     8430a3000000-8430a31fffff : 0002:01:03.1
       8430a3000000-8430a31fffff : thunder-nicvf
     8430a3200000-8430a33fffff : 0002:01:03.2
       8430a3200000-8430a33fffff : thunder-nicvf
     8430a3400000-8430a35fffff : 0002:01:03.3
       8430a3400000-8430a35fffff : thunder-nicvf
     8430a3600000-8430a37fffff : 0002:01:03.4
       8430a3600000-8430a37fffff : thunder-nicvf
     8430a3800000-8430a39fffff : 0002:01:03.5
       8430a3800000-8430a39fffff : thunder-nicvf
     8430a3a00000-8430a3bfffff : 0002:01:03.6
       8430a3a00000-8430a3bfffff : thunder-nicvf
     8430a3c00000-8430a3dfffff : 0002:01:03.7
       8430a3c00000-8430a3dfffff : thunder-nicvf
     8430a3e00000-8430a3ffffff : 0002:01:04.0
       8430a3e00000-8430a3ffffff : thunder-nicvf
     8430a4000000-8430a41fffff : 0002:01:04.1
       8430a4000000-8430a41fffff : thunder-nicvf
     8430a4200000-8430a43fffff : 0002:01:04.2
       8430a4200000-8430a43fffff : thunder-nicvf
     8430a4400000-8430a45fffff : 0002:01:04.3
       8430a4400000-8430a45fffff : thunder-nicvf
     8430a4600000-8430a47fffff : 0002:01:04.4
       8430a4600000-8430a47fffff : thunder-nicvf
   8430e0000000-8430efffffff : 0002:01:00.0
     8430e0000000-8430e01fffff : 0002:01:00.1
       8430e0000000-8430e01fffff : thunder-nicvf
     8430e0200000-8430e03fffff : 0002:01:00.2
       8430e0200000-8430e03fffff : thunder-nicvf
     8430e0400000-8430e05fffff : 0002:01:00.3
       8430e0400000-8430e05fffff : thunder-nicvf
     8430e0600000-8430e07fffff : 0002:01:00.4
       8430e0600000-8430e07fffff : thunder-nicvf
     8430e0800000-8430e09fffff : 0002:01:00.5
       8430e0800000-8430e09fffff : thunder-nicvf
     8430e0a00000-8430e0bfffff : 0002:01:00.6
       8430e0a00000-8430e0bfffff : thunder-nicvf
     8430e0c00000-8430e0dfffff : 0002:01:00.7
       8430e0c00000-8430e0dfffff : thunder-nicvf
     8430e0e00000-8430e0ffffff : 0002:01:01.0
       8430e0e00000-8430e0ffffff : thunder-nicvf
     8430e1000000-8430e11fffff : 0002:01:01.1
       8430e1000000-8430e11fffff : thunder-nicvf
     8430e1200000-8430e13fffff : 0002:01:01.2
       8430e1200000-8430e13fffff : thunder-nicvf
     8430e1400000-8430e15fffff : 0002:01:01.3
       8430e1400000-8430e15fffff : thunder-nicvf
     8430e1600000-8430e17fffff : 0002:01:01.4
       8430e1600000-8430e17fffff : thunder-nicvf
     8430e1800000-8430e19fffff : 0002:01:01.5
       8430e1800000-8430e19fffff : thunder-nicvf
     8430e1a00000-8430e1bfffff : 0002:01:01.6
       8430e1a00000-8430e1bfffff : thunder-nicvf
     8430e1c00000-8430e1dfffff : 0002:01:01.7
       8430e1c00000-8430e1dfffff : thunder-nicvf
     8430e1e00000-8430e1ffffff : 0002:01:02.0
       8430e1e00000-8430e1ffffff : thunder-nicvf
     8430e2000000-8430e21fffff : 0002:01:02.1
       8430e2000000-8430e21fffff : thunder-nicvf
     8430e2200000-8430e23fffff : 0002:01:02.2
       8430e2200000-8430e23fffff : thunder-nicvf
     8430e2400000-8430e25fffff : 0002:01:02.3
       8430e2400000-8430e25fffff : thunder-nicvf
     8430e2600000-8430e27fffff : 0002:01:02.4
       8430e2600000-8430e27fffff : thunder-nicvf
     8430e2800000-8430e29fffff : 0002:01:02.5
       8430e2800000-8430e29fffff : thunder-nicvf
     8430e2a00000-8430e2bfffff : 0002:01:02.6
       8430e2a00000-8430e2bfffff : thunder-nicvf
     8430e2c00000-8430e2dfffff : 0002:01:02.7
       8430e2c00000-8430e2dfffff : thunder-nicvf
     8430e2e00000-8430e2ffffff : 0002:01:03.0
       8430e2e00000-8430e2ffffff : thunder-nicvf
     8430e3000000-8430e31fffff : 0002:01:03.1
       8430e3000000-8430e31fffff : thunder-nicvf
     8430e3200000-8430e33fffff : 0002:01:03.2
       8430e3200000-8430e33fffff : thunder-nicvf
     8430e3400000-8430e35fffff : 0002:01:03.3
       8430e3400000-8430e35fffff : thunder-nicvf
     8430e3600000-8430e37fffff : 0002:01:03.4
       8430e3600000-8430e37fffff : thunder-nicvf
     8430e3800000-8430e39fffff : 0002:01:03.5
       8430e3800000-8430e39fffff : thunder-nicvf
     8430e3a00000-8430e3bfffff : 0002:01:03.6
       8430e3a00000-8430e3bfffff : thunder-nicvf
     8430e3c00000-8430e3dfffff : 0002:01:03.7
       8430e3c00000-8430e3dfffff : thunder-nicvf
     8430e3e00000-8430e3ffffff : 0002:01:04.0
       8430e3e00000-8430e3ffffff : thunder-nicvf
     8430e4000000-8430e41fffff : 0002:01:04.1
       8430e4000000-8430e41fffff : thunder-nicvf
     8430e4200000-8430e43fffff : 0002:01:04.2
       8430e4200000-8430e43fffff : thunder-nicvf
     8430e4400000-8430e45fffff : 0002:01:04.3
       8430e4400000-8430e45fffff : thunder-nicvf
     8430e4600000-8430e47fffff : 0002:01:04.4
       8430e4600000-8430e47fffff : thunder-nicvf
846000000000-847fffffffff : PCI Bus 0000:00
   846000000000-8467ffffffff : 0000:04:00.0
   846a00000000-846a000fffff : 0000:04:00.0
848000000000-848001ffffff : PCI ECAM
849000000000-849001ffffff : PCI ECAM
84a000000000-84a001ffffff : PCI ECAM
84b000000000-84b001ffffff : PCI ECAM
868000000000-87e023ffffff : PCI Bus 0000:00
   868000000000-8680001fffff : 0000:00:10.0
     868000000000-8680001fffff : xhci-hcd
   868000200000-8680002fffff : 0000:00:10.0
   869000000000-8690001fffff : 0000:00:11.0
     869000000000-8690001fffff : xhci-hcd
   869000200000-8690002fffff : 0000:00:11.0
   870000000000-8700007fffff : 0000:02:00.0
   870000f00000-870000ffffff : 0000:02:00.0
   87e005000000-87e0057fffff : 0000:01:01.3
     87e005000000-87e0057fffff : mdio_thunder
   87e006000000-87e0067fffff : 0000:01:00.1
   87e006f00000-87e006ffffff : 0000:01:00.1
   87e009000000-87e0097fffff : 0000:01:01.4
   87e009f00000-87e009ffffff : 0000:01:01.4
87e024000000-87e024000fff : ARMH0011:00
   87e024000000-87e024000fff : ARMH0011:00
87e025000000-87e025000fff : ARMH0011:01
   87e025000000-87e025000fff : ARMH0011:01
87e026000000-87e0bfffffff : PCI Bus 0000:00
   87e027000000-87e0277fffff : CAVA02A:00
   87e040000000-87e0400fffff : 0000:00:09.0
   87e050000000-87e0507fffff : 0000:01:06.0
   87e050f00000-87e050ffffff : 0000:01:06.0
   87e051000000-87e0517fffff : 0000:01:06.1
   87e051f00000-87e051ffffff : 0000:01:06.1
   87e052000000-87e0527fffff : 0000:01:06.2
   87e052f00000-87e052ffffff : 0000:01:06.2
   87e053000000-87e0537fffff : 0000:01:06.3
   87e053f00000-87e053ffffff : 0000:01:06.3
   87e054000000-87e0547fffff : 0000:01:06.4
   87e054f00000-87e054ffffff : 0000:01:06.4
   87e055000000-87e0557fffff : 0000:01:06.5
   87e055f00000-87e055ffffff : 0000:01:06.5
   87e056000000-87e0567fffff : 0000:01:06.6
   87e056f00000-87e056ffffff : 0000:01:06.6
   87e057000000-87e0577fffff : 0000:01:06.7
   87e057f00000-87e057ffffff : 0000:01:06.7
   87e058000000-87e0587fffff : 0000:01:07.0
   87e058f00000-87e058ffffff : 0000:01:07.0
   87e059000000-87e0597fffff : 0000:01:07.1
   87e059f00000-87e059ffffff : 0000:01:07.1
   87e05a000000-87e05a7fffff : 0000:01:07.2
   87e05af00000-87e05affffff : 0000:01:07.2
   87e05b000000-87e05b7fffff : 0000:01:07.3
   87e05bf00000-87e05bffffff : 0000:01:07.3
   87e05c000000-87e05c7fffff : 0000:01:07.4
   87e05cf00000-87e05cffffff : 0000:01:07.4
   87e05d000000-87e05d7fffff : 0000:01:07.5
   87e05df00000-87e05dffffff : 0000:01:07.5
   87e05e000000-87e05e7fffff : 0000:01:07.6
   87e05ef00000-87e05effffff : 0000:01:07.6
   87e05f000000-87e05f7fffff : 0000:01:07.7
   87e05ff00000-87e05fffffff : 0000:01:07.7
   87e088000000-87e0887fffff : 0000:01:0a.0
   87e088f00000-87e088ffffff : 0000:01:0a.0
   87e089000000-87e0897fffff : 0000:01:0a.1
   87e089f00000-87e089ffffff : 0000:01:0a.1
   87e08a000000-87e08a7fffff : 0000:01:0a.2
   87e08af00000-87e08affffff : 0000:01:0a.2
   87e08b000000-87e08b7fffff : 0000:01:0a.3
   87e08bf00000-87e08bffffff : 0000:01:0a.3
87e0c0000000-87e0c0ffffff : PCI Bus 0004:1f
   87e0c0f00000-87e0c0ffffff : 0004:1f:00.0
87e0c2000000-87e0c2ffffff : PCI Bus 0006:8f
   87e0c2f00000-87e0c2ffffff : 0006:8f:00.0
87e0c6000000-87ffffffffff : PCI Bus 0000:00
   87e0d0000000-87e0d07fffff : 0000:01:09.0
   87e0d0f00000-87e0d0ffffff : 0000:01:09.0
   87e0d1000000-87e0d17fffff : 0000:01:09.1
   87e0d1f00000-87e0d1ffffff : 0000:01:09.1
   87e0d2000000-87e0d27fffff : 0000:01:09.2
   87e0d2f00000-87e0d2ffffff : 0000:01:09.2
   87e0d3000000-87e0d37fffff : 0000:01:09.3
   87e0d3f00000-87e0d3ffffff : 0000:01:09.3
   87e0d4000000-87e0d47fffff : 0000:01:09.4
   87e0d4f00000-87e0d4ffffff : 0000:01:09.4
   87e0e0000000-87e0e03fffff : 0000:01:10.0
     87e0e0000000-87e0e03fffff : thunder-BGX
   87e0e0400000-87e0e07fffff : 0000:01:10.0
     87e0e0400000-87e0e07fffff : thunder-BGX
   87e0e1000000-87e0e13fffff : 0000:01:10.1
     87e0e1000000-87e0e13fffff : thunder-BGX
   87e0e1400000-87e0e17fffff : 0000:01:10.1
     87e0e1400000-87e0e17fffff : thunder-BGX
   87e0fc000000-87e0fc0fffff : 0000:01:00.0
   87e0fcf00000-87e0fcffffff : 0000:01:00.0
88001f000000-880057ffffff : PCI ECAM
881010000000-881fffffffff : PCI Bus 0004:1f
   881010000000-8810117fffff : PCI Bus 0004:20
     881010000000-8810117fffff : PCI Bus 0004:21
       881010000000-881010ffffff : 0004:21:00.0
       881011000000-88101101ffff : 0004:21:00.0
   881011800000-88101180ffff : 0004:1f:00.0
882000000000-882fffffffff : PCI Bus 0004:1f
88808f000000-8880c7ffffff : PCI ECAM
889010000000-889fffffffff : PCI Bus 0006:8f
   889010000000-8890107fffff : PCI Bus 0006:90
     889010000000-8890103fffff : 0006:90:00.0
     889010400000-8890107fffff : 0006:90:00.0
   889010800000-88901080ffff : 0006:8f:00.0
88a000000000-88afffffffff : PCI Bus 0006:8f
   88a000000000-88a0000fffff : PCI Bus 0006:90
     88a000000000-88a00007ffff : 0006:90:00.0
       88a000000000-88a00007ffff : ixgbe
     88a000080000-88a000083fff : 0006:90:00.0
       88a000080000-88a000083fff : ixgbe

Thanks,
Tomasz